[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:29 dist-test-slave-8w9v krb5kdc[18941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:29 dist-test-slave-8w9v krb5kdc[18941](info): set up 2 sockets
Aug 06 18:34:29 dist-test-slave-8w9v krb5kdc[18941](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 18:34:30.196630 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.009s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:34:30 dist-test-slave-8w9v krb5kdc[18941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:34:30.328229 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:46409
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:46409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:30.444569 18954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:30.444926 18954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:30.445010 18954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:30.449223 18954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:30.449334 18954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:30.449373 18954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:30.449409 18954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:30.449445 18954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:30.455009 18954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:46409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:46409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:30.456391 18954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:30.457576 18954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:30.467530 18960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:30.467525 18962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:30.467525 18959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:30.467939 18954 server_base.cc:1034] running on GCE node
I20260806 18:34:30.468672 18954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:30.469921 18954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:30.471153 18954 hybrid_clock.cc:648] HybridClock initialized: now 1786041270471136 us; error 96 us; skew 500 ppm
Aug 06 18:34:30 dist-test-slave-8w9v krb5kdc[18941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041270, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:30.474360 18954 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:34:30.475431 18954 webserver.cc:460] Webserver started at http://127.18.125.190:40727/ using document root <none> and password file <none>
I20260806 18:34:30.476094 18954 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:30.476253 18954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:30.476531 18954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:30.478837 18954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "f06aed8011464641abaa6aa34c305fab"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-8w9v"
server_key: "cd49f7c568a19577debf6a98cbaf91d5"
server_key_iv: "117ed32885c925406754e53ac753ef31"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.479463 18954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "f06aed8011464641abaa6aa34c305fab"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-8w9v"
server_key: "cd49f7c568a19577debf6a98cbaf91d5"
server_key_iv: "117ed32885c925406754e53ac753ef31"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.483772 18954 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:34:30.486614 18970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:30.488001 18954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:30.488209 18954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "f06aed8011464641abaa6aa34c305fab"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-8w9v"
server_key: "cd49f7c568a19577debf6a98cbaf91d5"
server_key_iv: "117ed32885c925406754e53ac753ef31"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.488351 18954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:30.507730 18954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:30.511161 18954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:30.511401 18954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:30.519385 18954 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:46409
I20260806 18:34:30.519414 19024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:46409 every 8 connection(s)
I20260806 18:34:30.520417 18954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:30.523290 19025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:30.525900 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 18954
I20260806 18:34:30.526072 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:30.527822 18934 external_mini_cluster.cc:1428] Setting key e763ddef428bbf5df49540b2e185bbff
I20260806 18:34:30.531052 19025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab: Bootstrap starting.
I20260806 18:34:30.534256 19025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:30.535274 19025 log.cc:824] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:30.537827 19025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab: No bootstrap required, opened a new log
I20260806 18:34:30.542845 19025 raft_consensus.cc:348] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06aed8011464641abaa6aa34c305fab" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46409 } }
I20260806 18:34:30.543112 19025 raft_consensus.cc:374] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:30.543149 19025 raft_consensus.cc:729] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06aed8011464641abaa6aa34c305fab, State: Initialized, Role: FOLLOWER
I20260806 18:34:30.543742 19025 consensus_queue.cc:260] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06aed8011464641abaa6aa34c305fab" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46409 } }
I20260806 18:34:30.543934 19025 raft_consensus.cc:388] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:30.544013 19025 raft_consensus.cc:482] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:30.544142 19025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:30.545433 19025 raft_consensus.cc:504] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06aed8011464641abaa6aa34c305fab" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46409 } }
I20260806 18:34:30.545850 19025 leader_election.cc:302] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f06aed8011464641abaa6aa34c305fab; no voters:
I20260806 18:34:30.546259 19025 leader_election.cc:288] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:30.546428 19031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:30.546711 19031 raft_consensus.cc:686] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [term 1 LEADER]: Becoming Leader. State: Replica: f06aed8011464641abaa6aa34c305fab, State: Running, Role: LEADER
I20260806 18:34:30.547067 19031 consensus_queue.cc:237] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06aed8011464641abaa6aa34c305fab" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46409 } }
I20260806 18:34:30.547770 19025 sys_catalog.cc:564] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:30.549238 19032 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f06aed8011464641abaa6aa34c305fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06aed8011464641abaa6aa34c305fab" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46409 } } }
I20260806 18:34:30.549408 19032 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:30.549238 19033 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [sys.catalog]: SysCatalogTable state changed. Reason: New leader f06aed8011464641abaa6aa34c305fab. Latest consensus state: current_term: 1 leader_uuid: "f06aed8011464641abaa6aa34c305fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06aed8011464641abaa6aa34c305fab" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46409 } } }
I20260806 18:34:30.549526 19033 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:30.549914 19040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 18:34:30 dist-test-slave-8w9v krb5kdc[18941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:30.553731 19040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:30.558962 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.531897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57012 (local address 127.18.125.190:46409)
0806 18:34:30.532508 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.532530 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.533649 (+ 1119us) server_negotiation.cc:407] Connection header received
0806 18:34:30.538143 (+ 4494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.538163 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.538531 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.538836 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:30.542198 (+ 3362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.543319 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.545133 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.545700 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.545844 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.551593 (+ 5749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:30.551600 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:30.551617 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:30.551657 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:30.554037 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:30.554908 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:30.554915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:30.554922 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:30.555093 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:30.555589 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:30.555593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:30.555595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:30.556363 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:30.556592 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.557343 (+ 751us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.557678 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":202,"threads_started":1}
I20260806 18:34:30.561662 19040 catalog_manager.cc:1347] Generated new cluster ID: 0d74dec5b3d54171a91e07da16db736a
I20260806 18:34:30.561751 19040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:30.574483 19040 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:30.575532 19040 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:30.595841 19040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f06aed8011464641abaa6aa34c305fab: Generated new TSK 0
I20260806 18:34:30.596679 19040 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:34:30.631093 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46409
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:30.758358 19053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:30.758638 19053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:30.758687 19053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:30.762812 19053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:30.762887 19053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:30.762991 19053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:34:30.769781 19053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46409
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:30.771031 19053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:30.772197 19053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:30.784664 19059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:30.784664 19061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:30.784936 19058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:30.784911 19053 server_base.cc:1034] running on GCE node
I20260806 18:34:30.785593 19053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:30.786369 19053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:30.787569 19053 hybrid_clock.cc:648] HybridClock initialized: now 1786041270787554 us; error 35 us; skew 500 ppm
Aug 06 18:34:30 dist-test-slave-8w9v krb5kdc[18941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041270, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:30.791234 19053 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:34:30.792526 19053 webserver.cc:460] Webserver started at http://127.18.125.129:40321/ using document root <none> and password file <none>
I20260806 18:34:30.793233 19053 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:30.793340 19053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:30.793605 19053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:30.795923 19053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "29884047dba14889a6deac493927ebb4"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-8w9v"
server_key: "59ec36f26b32ec4a3e1d64486813a71f"
server_key_iv: "518854280170a13e8c624e19dbcf0f72"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.796649 19053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "29884047dba14889a6deac493927ebb4"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-8w9v"
server_key: "59ec36f26b32ec4a3e1d64486813a71f"
server_key_iv: "518854280170a13e8c624e19dbcf0f72"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.803519 19053 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 18:34:30.807966 19068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:30.809547 19053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:30.809737 19053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "29884047dba14889a6deac493927ebb4"
format_stamp: "Formatted at 2026-08-06 18:34:30 on dist-test-slave-8w9v"
server_key: "59ec36f26b32ec4a3e1d64486813a71f"
server_key_iv: "518854280170a13e8c624e19dbcf0f72"
server_key_version: "encryptionkey@0"
I20260806 18:34:30.809898 19053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:30.833568 19053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:30.837100 19053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:30.837357 19053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:30.838028 19053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:30.839039 19053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:30.839118 19053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:30.839202 19053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:30.839249 19053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:30.849223 19053 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:41225
I20260806 18:34:30.849299 19187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:41225 every 8 connection(s)
I20260806 18:34:30.850224 19053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:30.859756 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 19053
I20260806 18:34:30.859920 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:30.860285 18934 external_mini_cluster.cc:1428] Setting key 73c61cd84118c66014374e6242398d35
Aug 06 18:34:30 dist-test-slave-8w9v krb5kdc[18941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041270, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:30.868381 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.853495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:42721 (local address 127.18.125.190:46409)
0806 18:34:30.853704 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:30.853709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:30.854830 (+ 1121us) server_negotiation.cc:407] Connection header received
0806 18:34:30.856568 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:30.856571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:30.856629 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:30.856723 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:30.858158 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.858747 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.860210 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.860599 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.860735 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.863866 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:30.863869 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:30.863871 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:30.863902 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:30.865527 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:30.866196 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:30.866200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:30.866202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:30.866296 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:30.866995 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:30.866998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:30.867000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:30.867235 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:30.867363 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:30.867999 (+ 636us) server_negotiation.cc:299] Negotiation successful
0806 18:34:30.868197 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 18:34:30.869343 19190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:30.853974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:46409 (local address 127.18.125.129:42721)
0806 18:34:30.854616 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:30.854670 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:30.856036 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:30.856934 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:30.856943 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:30.857372 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:30.857953 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.857969 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.858929 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.858947 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.859984 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:30.860002 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:30.860806 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:30.860812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:30.861082 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:30.861820 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:30.861840 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:30.863646 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:30.865698 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:30.865713 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:30.865732 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:30.866075 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:30.866402 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:30.866405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:30.866407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:30.866836 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:30.867368 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:30.867378 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:30.867710 (+ 332us) client_negotiation.cc:769] Sending connection context
0806 18:34:30.867951 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 18:34:30.868400 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":185,"threads_started":1}
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:34:30.870919 19188 heartbeater.cc:347] Connected to a master server at 127.18.125.190:46409
I20260806 18:34:30.871248 19188 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:30.871886 19188 heartbeater.cc:511] Master 127.18.125.190:46409 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
I20260806 18:34:30.873862 18988 ts_manager.cc:194] Registered new tserver with Master: 29884047dba14889a6deac493927ebb4 (127.18.125.129:41225)
I20260806 18:34:30.875159 18988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:42721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:34:30.893491 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46409
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:31.018723 19194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:31.019042 19194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:31.019093 19194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:31.023190 19194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:31.023277 19194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:31.023345 19194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:34:31.029574 19194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46409
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:31.030731 19194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:31.031770 19194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:31.042235 19200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:31.042270 19199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:31.042414 19194 server_base.cc:1034] running on GCE node
W20260806 18:34:31.042584 19202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:31.043049 19194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:31.043723 19194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:31.044917 19194 hybrid_clock.cc:648] HybridClock initialized: now 1786041271044855 us; error 59 us; skew 500 ppm
Aug 06 18:34:31 dist-test-slave-8w9v krb5kdc[18941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041271, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:31.047894 19194 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:34:31.049024 19194 webserver.cc:460] Webserver started at http://127.18.125.130:44319/ using document root <none> and password file <none>
I20260806 18:34:31.049739 19194 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:31.049844 19194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:31.050113 19194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:31.052459 19194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "deae287c9b86473f924140f346c7641a"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-8w9v"
server_key: "1f1f58cdd96a7c2ee9021cd32461f8bc"
server_key_iv: "f0b18c110a2ebdcef17dad59f2a11b76"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.053102 19194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "deae287c9b86473f924140f346c7641a"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-8w9v"
server_key: "1f1f58cdd96a7c2ee9021cd32461f8bc"
server_key_iv: "f0b18c110a2ebdcef17dad59f2a11b76"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.057538 19194 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:34:31.060799 19209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.062320 19194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:31.062490 19194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "deae287c9b86473f924140f346c7641a"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-8w9v"
server_key: "1f1f58cdd96a7c2ee9021cd32461f8bc"
server_key_iv: "f0b18c110a2ebdcef17dad59f2a11b76"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.062638 19194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:31.085105 19194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:31.088560 19194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:31.088804 19194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:31.089504 19194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:31.090462 19194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:31.090516 19194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.090564 19194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:31.090624 19194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.101963 19194 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:46483
I20260806 18:34:31.101984 19326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:46483 every 8 connection(s)
I20260806 18:34:31.103008 19194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:31.112186 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 19194
I20260806 18:34:31.112344 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:31.112644 18934 external_mini_cluster.cc:1428] Setting key 353572e7f3405604c32836f90e4bd296
Aug 06 18:34:31 dist-test-slave-8w9v krb5kdc[18941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041271, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:31.119149 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.105116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:53161 (local address 127.18.125.190:46409)
0806 18:34:31.105267 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.105272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.106077 (+ 805us) server_negotiation.cc:407] Connection header received
0806 18:34:31.107557 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.107561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.107629 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.107749 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.109154 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.109740 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.111037 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.111290 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.111419 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:31.114619 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.114622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.114625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:31.114651 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.116641 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:31.117223 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:31.117228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:31.117232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:31.117344 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:31.117821 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:31.117824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:31.117825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:31.118015 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.118096 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.118888 (+ 792us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.119006 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:34:31.120676 19329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.105422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:46409 (local address 127.18.125.130:53161)
0806 18:34:31.105908 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.105958 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:31.107323 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:31.107946 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:31.107954 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:31.108459 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:31.108986 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:31.109001 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.109906 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.109910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:31.110891 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:31.110902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.111471 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.111475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:31.111665 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:31.112456 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:31.112486 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:31.114434 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:31.116782 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:31.116789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:31.116804 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:31.117106 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:31.117449 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:31.117452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:31.117454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:31.117697 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:31.118135 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:31.118146 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:31.118543 (+ 397us) client_negotiation.cc:769] Sending connection context
0806 18:34:31.118823 (+ 280us) client_negotiation.cc:240] Negotiation successful
0806 18:34:31.119298 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":125,"threads_started":1}
I20260806 18:34:31.122004 19327 heartbeater.cc:347] Connected to a master server at 127.18.125.190:46409
I20260806 18:34:31.122267 19327 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:31.122825 19327 heartbeater.cc:511] Master 127.18.125.190:46409 requested a full tablet report, sending...
I20260806 18:34:31.123940 18988 ts_manager.cc:194] Registered new tserver with Master: deae287c9b86473f924140f346c7641a (127.18.125.130:46483)
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:34:31.124626 18988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:53161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:34:31.145449 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46409
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:31.268524 19332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:31.268863 19332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:31.268965 19332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:31.273283 19332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:31.273386 19332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:31.273486 19332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:34:31.279325 19332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46409
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:31.280661 19332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:31.281787 19332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:31.292510 19337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:31.292534 19342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:31.292809 19339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:31.293072 19332 server_base.cc:1034] running on GCE node
I20260806 18:34:31.293627 19332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:31.294344 19332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:31.295518 19332 hybrid_clock.cc:648] HybridClock initialized: now 1786041271295500 us; error 32 us; skew 500 ppm
Aug 06 18:34:31 dist-test-slave-8w9v krb5kdc[18941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041271, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:31.298503 19332 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:34:31.299505 19332 webserver.cc:460] Webserver started at http://127.18.125.131:46837/ using document root <none> and password file <none>
I20260806 18:34:31.300268 19332 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:31.300349 19332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:31.300551 19332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:31.302754 19332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "305abd84b4f64de6aa77939a4053eafa"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-8w9v"
server_key: "9a26a3c4db7af326115519201f4f57a5"
server_key_iv: "3d4f669b205d658d225e2aeeeceb48e2"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.303323 19332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "305abd84b4f64de6aa77939a4053eafa"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-8w9v"
server_key: "9a26a3c4db7af326115519201f4f57a5"
server_key_iv: "3d4f669b205d658d225e2aeeeceb48e2"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.307750 19332 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:34:31.311120 19349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.313023 19332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:31.313233 19332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "305abd84b4f64de6aa77939a4053eafa"
format_stamp: "Formatted at 2026-08-06 18:34:31 on dist-test-slave-8w9v"
server_key: "9a26a3c4db7af326115519201f4f57a5"
server_key_iv: "3d4f669b205d658d225e2aeeeceb48e2"
server_key_version: "encryptionkey@0"
I20260806 18:34:31.313398 19332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:31.343380 19332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:31.346908 19332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:31.347151 19332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:31.347952 19332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:31.349062 19332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:31.349125 19332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.349197 19332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:31.349234 19332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:31.359249 19332 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:37351
I20260806 18:34:31.359272 19469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:37351 every 8 connection(s)
I20260806 18:34:31.360253 19332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:31.364001 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 19332
I20260806 18:34:31.364202 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:31.364563 18934 external_mini_cluster.cc:1428] Setting key b00c89eef150d90c3b7f330a35657d8f
Aug 06 18:34:31 dist-test-slave-8w9v krb5kdc[18941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041271, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:31.376224 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.362521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:52721 (local address 127.18.125.190:46409)
0806 18:34:31.362724 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.362732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.363587 (+ 855us) server_negotiation.cc:407] Connection header received
0806 18:34:31.365352 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.365355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.365419 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.365533 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.367021 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.367741 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.368994 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.369234 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.369353 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:31.372005 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.372008 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.372010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:31.372037 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.373783 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:31.374445 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:31.374449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:31.374450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:31.374540 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:31.374934 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:31.374937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:31.374939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:31.375150 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.375278 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.375849 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.376006 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 18:34:31.377106 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.362815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:46409 (local address 127.18.125.131:52721)
0806 18:34:31.363404 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:31.363451 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:31.365122 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:31.365714 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:31.365721 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:31.366272 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:31.366859 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:31.366880 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.367872 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.367876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:31.368862 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:31.368873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.369407 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.369412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:31.369582 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:31.370228 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:31.370249 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:31.371774 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:31.373953 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:31.373962 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:31.373984 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:31.374315 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:31.374645 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:31.374648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:31.374650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:31.374823 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:31.375271 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:31.375279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:31.375568 (+ 289us) client_negotiation.cc:769] Sending connection context
0806 18:34:31.375801 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 18:34:31.376221 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":141,"threads_started":1}
I20260806 18:34:31.378546 19470 heartbeater.cc:347] Connected to a master server at 127.18.125.190:46409
I20260806 18:34:31.378856 19470 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:31.379626 19470 heartbeater.cc:511] Master 127.18.125.190:46409 requested a full tablet report, sending...
I20260806 18:34:31.380826 18988 ts_manager.cc:194] Registered new tserver with Master: 305abd84b4f64de6aa77939a4053eafa (127.18.125.131:37351)
I20260806 18:34:31.381429 18988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:52721
I20260806 18:34:31.392441 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:34:31 dist-test-slave-8w9v krb5kdc[18941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041271, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:34:31 dist-test-slave-8w9v krb5kdc[18941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041271, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:34:31.422173 19478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.410749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56300 (local address 127.18.125.129:41225)
0806 18:34:31.411108 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.411115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.411142 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:34:31.411266 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.411271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.411489 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.411618 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.412436 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.413215 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.414229 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.414662 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.414766 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:31.417889 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.417900 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.417915 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:31.417946 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.419805 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:31.420466 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:31.420472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:31.420474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:31.420570 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:31.420869 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:31.420873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:31.420874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:31.421197 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.421538 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.421759 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.421995 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":99,"threads_started":1}
W20260806 18:34:31.423322 19097 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56300
Aug 06 18:34:31 dist-test-slave-8w9v krb5kdc[18941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:34:31 dist-test-slave-8w9v krb5kdc[18941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:34:31.451386 19478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:31.442504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56302 (local address 127.18.125.129:41225)
0806 18:34:31.442657 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:31.442661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:31.442775 (+ 114us) server_negotiation.cc:407] Connection header received
0806 18:34:31.442904 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:31.442907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:31.442952 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:31.443036 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:31.443810 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.444453 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.445292 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:31.445511 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:31.445593 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:31.448237 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:31.448241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:31.448243 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:31.448293 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:31.449825 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:31.450234 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:31.450237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:31.450239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:31.450319 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:31.450572 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:31.450575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:31.450576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:31.450746 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:31.450851 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:31.451096 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 18:34:31.451210 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:34:31.452550 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 19053
I20260806 18:34:31.460546 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 19194
I20260806 18:34:31.467535 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 19332
I20260806 18:34:31.474293 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 18954
2026-08-06T18:34:31Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2412 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:31 dist-test-slave-8w9v krb5kdc[19485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:31 dist-test-slave-8w9v krb5kdc[19485](info): set up 2 sockets
Aug 06 18:34:31 dist-test-slave-8w9v krb5kdc[19485](info): commencing operation
krb5kdc: starting...
W20260806 18:34:32.580492 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:34:32 dist-test-slave-8w9v krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:34:32.712654 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:36725
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:36725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:32.843338 19499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:32.843652 19499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:32.843698 19499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:32.847872 19499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:32.847957 19499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:32.847977 19499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:32.847994 19499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:32.848063 19499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:32.853868 19499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:36725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:36725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:32.855088 19499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:32.856262 19499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:32.865640 19507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:32.865772 19504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:32.865640 19505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:32.866087 19499 server_base.cc:1034] running on GCE node
I20260806 18:34:32.866705 19499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:32.867901 19499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:32.869107 19499 hybrid_clock.cc:648] HybridClock initialized: now 1786041272869085 us; error 63 us; skew 500 ppm
Aug 06 18:34:32 dist-test-slave-8w9v krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041272, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:32.872078 19499 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:34:32.873150 19499 webserver.cc:460] Webserver started at http://127.18.125.190:33697/ using document root <none> and password file <none>
I20260806 18:34:32.873796 19499 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:32.873891 19499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:32.874141 19499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:32.876410 19499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "6be36bcfc6f746d78105a47c06ccfedf"
format_stamp: "Formatted at 2026-08-06 18:34:32 on dist-test-slave-8w9v"
server_key: "f4f9e7f0ea215cd5514395565e7ca5cd"
server_key_iv: "a190d42b9169bbf34451a54f794074a3"
server_key_version: "encryptionkey@0"
I20260806 18:34:32.877017 19499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "6be36bcfc6f746d78105a47c06ccfedf"
format_stamp: "Formatted at 2026-08-06 18:34:32 on dist-test-slave-8w9v"
server_key: "f4f9e7f0ea215cd5514395565e7ca5cd"
server_key_iv: "a190d42b9169bbf34451a54f794074a3"
server_key_version: "encryptionkey@0"
I20260806 18:34:32.881016 19499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:34:32.883613 19514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:32.884964 19499 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:32.885114 19499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "6be36bcfc6f746d78105a47c06ccfedf"
format_stamp: "Formatted at 2026-08-06 18:34:32 on dist-test-slave-8w9v"
server_key: "f4f9e7f0ea215cd5514395565e7ca5cd"
server_key_iv: "a190d42b9169bbf34451a54f794074a3"
server_key_version: "encryptionkey@0"
I20260806 18:34:32.885241 19499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:32.920023 19499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:32.923804 19499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:32.924053 19499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:32.931722 19499 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:36725
I20260806 18:34:32.931751 19570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:36725 every 8 connection(s)
I20260806 18:34:32.932823 19499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:32.936091 19571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:32.941816 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 19499
I20260806 18:34:32.941926 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:32.942245 18934 external_mini_cluster.cc:1428] Setting key ded3cddac00b76ff7b69bf7c74568fe7
I20260806 18:34:32.943532 19571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf: Bootstrap starting.
I20260806 18:34:32.946178 19571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:32.947161 19571 log.cc:824] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:32.949679 19571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf: No bootstrap required, opened a new log
Aug 06 18:34:32 dist-test-slave-8w9v krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:32.954538 19571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be36bcfc6f746d78105a47c06ccfedf" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36725 } }
I20260806 18:34:32.954828 19571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:32.954890 19571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6be36bcfc6f746d78105a47c06ccfedf, State: Initialized, Role: FOLLOWER
I20260806 18:34:32.955416 19571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be36bcfc6f746d78105a47c06ccfedf" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36725 } }
I20260806 18:34:32.955559 19571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:32.955647 19571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:32.955750 19571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:32.956902 19571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be36bcfc6f746d78105a47c06ccfedf" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36725 } }
I20260806 18:34:32.957289 19571 leader_election.cc:302] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6be36bcfc6f746d78105a47c06ccfedf; no voters:
I20260806 18:34:32.957706 19571 leader_election.cc:288] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:32.957793 19576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:32.958035 19576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [term 1 LEADER]: Becoming Leader. State: Replica: 6be36bcfc6f746d78105a47c06ccfedf, State: Running, Role: LEADER
I20260806 18:34:32.958595 19576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be36bcfc6f746d78105a47c06ccfedf" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36725 } }
I20260806 18:34:32.959208 19571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:32.959648 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:32.943898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49430 (local address 127.18.125.190:36725)
0806 18:34:32.944425 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:32.944452 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:32.944492 (+ 40us) server_negotiation.cc:407] Connection header received
0806 18:34:32.945194 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:32.945211 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:32.945553 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:32.945846 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:32.946906 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.948008 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.949334 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:32.950302 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:32.950470 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:32.953649 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:32.953655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:32.953672 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:32.953705 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:32.955773 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:32.956437 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:32.956444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:32.956450 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:32.956606 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:32.956944 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:32.956948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:32.956950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:32.957493 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:32.957719 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:32.958407 (+ 688us) server_negotiation.cc:299] Negotiation successful
0806 18:34:32.958732 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":221,"threads_started":1}
I20260806 18:34:32.959911 19576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6be36bcfc6f746d78105a47c06ccfedf. Latest consensus state: current_term: 1 leader_uuid: "6be36bcfc6f746d78105a47c06ccfedf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be36bcfc6f746d78105a47c06ccfedf" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36725 } } }
I20260806 18:34:32.960063 19576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:32.960537 19583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:32.960662 19577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6be36bcfc6f746d78105a47c06ccfedf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be36bcfc6f746d78105a47c06ccfedf" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36725 } } }
I20260806 18:34:32.960839 19577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:32.966549 19583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:32.973922 19583 catalog_manager.cc:1347] Generated new cluster ID: 35840d87f97949949c01001eb92cb8da
I20260806 18:34:32.974025 19583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:32.990703 19583 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:32.991732 19583 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:32.997327 19583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6be36bcfc6f746d78105a47c06ccfedf: Generated new TSK 0
I20260806 18:34:32.998179 19583 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:34:33.036511 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36725
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:33.155370 19597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:33.155721 19597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:33.155808 19597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:33.160260 19597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:33.160359 19597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:33.160476 19597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:34:33.166167 19597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:33.167390 19597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:33.168519 19597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:33.179206 19602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:33.179206 19605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:33.179207 19603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:33.179610 19597 server_base.cc:1034] running on GCE node
I20260806 18:34:33.180089 19597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:33.180822 19597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:33.182029 19597 hybrid_clock.cc:648] HybridClock initialized: now 1786041273182007 us; error 33 us; skew 500 ppm
Aug 06 18:34:33 dist-test-slave-8w9v krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041273, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:33.185125 19597 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:34:33.186066 19597 webserver.cc:460] Webserver started at http://127.18.125.129:44337/ using document root <none> and password file <none>
I20260806 18:34:33.186676 19597 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:33.186774 19597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:33.187032 19597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:33.189397 19597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "b4215b39f333494086178ed8b62f116d"
format_stamp: "Formatted at 2026-08-06 18:34:33 on dist-test-slave-8w9v"
server_key: "a9566d2332d6430d8a6a87f86bb05c0a"
server_key_iv: "c1ffedb044a619e0ad81e27ad042d4c4"
server_key_version: "encryptionkey@0"
I20260806 18:34:33.190045 19597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "b4215b39f333494086178ed8b62f116d"
format_stamp: "Formatted at 2026-08-06 18:34:33 on dist-test-slave-8w9v"
server_key: "a9566d2332d6430d8a6a87f86bb05c0a"
server_key_iv: "c1ffedb044a619e0ad81e27ad042d4c4"
server_key_version: "encryptionkey@0"
I20260806 18:34:33.193974 19597 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:33.196650 19612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:33.197873 19597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:33.198045 19597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "b4215b39f333494086178ed8b62f116d"
format_stamp: "Formatted at 2026-08-06 18:34:33 on dist-test-slave-8w9v"
server_key: "a9566d2332d6430d8a6a87f86bb05c0a"
server_key_iv: "c1ffedb044a619e0ad81e27ad042d4c4"
server_key_version: "encryptionkey@0"
I20260806 18:34:33.198185 19597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:33.224383 19597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:33.227932 19597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:33.228223 19597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:33.228886 19597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:33.229761 19597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:33.229835 19597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:33.229910 19597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:33.229955 19597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:33.240932 19597 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:38171
I20260806 18:34:33.240960 19730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:38171 every 8 connection(s)
I20260806 18:34:33.242112 19597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:33.244571 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 19597
I20260806 18:34:33.244699 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:33.244990 18934 external_mini_cluster.cc:1428] Setting key 837c470918fc6927a040add2419a7620
Aug 06 18:34:33 dist-test-slave-8w9v krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041273, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:34:33.260049 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.244444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:37247 (local address 127.18.125.190:36725)
0806 18:34:33.244644 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:33.244650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:33.245790 (+ 1140us) server_negotiation.cc:407] Connection header received
0806 18:34:33.246977 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:33.246980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:33.247041 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:33.247143 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:33.248594 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.249213 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.251214 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.251485 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.251670 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.255410 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:33.255414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:33.255417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:33.255452 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:33.257254 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.257883 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.257886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.257887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.257993 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.258461 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.258463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.258464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.258665 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:33.258746 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:33.259646 (+ 900us) server_negotiation.cc:299] Negotiation successful
0806 18:34:33.259789 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
I20260806 18:34:33.261158 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.244884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:36725 (local address 127.18.125.129:37247)
0806 18:34:33.245569 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:33.245648 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:33.246751 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:33.247330 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:33.247338 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:33.247765 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:33.248379 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.248406 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.249371 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.249378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.251028 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.251046 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.251746 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.251754 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.252017 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.252933 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:33.252964 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:33.255117 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:33.257401 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:33.257410 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:33.257426 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:33.257732 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:33.258145 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:33.258148 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:33.258149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:33.258353 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:33.258835 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:33.258842 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:33.259379 (+ 537us) client_negotiation.cc:769] Sending connection context
0806 18:34:33.259562 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 18:34:33.259875 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":205,"threads_started":1}
I20260806 18:34:33.262728 19731 heartbeater.cc:347] Connected to a master server at 127.18.125.190:36725
I20260806 18:34:33.263056 19731 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:33.263772 19731 heartbeater.cc:511] Master 127.18.125.190:36725 requested a full tablet report, sending...
I20260806 18:34:33.265686 19532 ts_manager.cc:194] Registered new tserver with Master: b4215b39f333494086178ed8b62f116d (127.18.125.129:38171)
I20260806 18:34:33.266937 19532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:37247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:34:33.279986 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36725
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:33.404567 19736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:33.405006 19736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:33.405107 19736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:33.409461 19736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:33.409580 19736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:33.409691 19736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:34:33.415504 19736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:33.416818 19736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:33.417968 19736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:33.428857 19744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:33.428850 19741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:33.428925 19742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:33.429066 19736 server_base.cc:1034] running on GCE node
I20260806 18:34:33.429674 19736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:33.430425 19736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:33.431634 19736 hybrid_clock.cc:648] HybridClock initialized: now 1786041273431619 us; error 33 us; skew 500 ppm
Aug 06 18:34:33 dist-test-slave-8w9v krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041273, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:33.434752 19736 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:34:33.435792 19736 webserver.cc:460] Webserver started at http://127.18.125.130:33957/ using document root <none> and password file <none>
I20260806 18:34:33.436579 19736 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:33.436683 19736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:33.436946 19736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:33.439167 19736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "2e0313f23110469b91db77bb91b6630c"
format_stamp: "Formatted at 2026-08-06 18:34:33 on dist-test-slave-8w9v"
server_key: "64621f7e856de9b9da2b67dd32ab33e5"
server_key_iv: "a6f78293658831f20959b810d29f97ce"
server_key_version: "encryptionkey@0"
I20260806 18:34:33.439776 19736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "2e0313f23110469b91db77bb91b6630c"
format_stamp: "Formatted at 2026-08-06 18:34:33 on dist-test-slave-8w9v"
server_key: "64621f7e856de9b9da2b67dd32ab33e5"
server_key_iv: "a6f78293658831f20959b810d29f97ce"
server_key_version: "encryptionkey@0"
I20260806 18:34:33.443791 19736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:34:33.446748 19751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:33.448278 19736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:33.448436 19736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "2e0313f23110469b91db77bb91b6630c"
format_stamp: "Formatted at 2026-08-06 18:34:33 on dist-test-slave-8w9v"
server_key: "64621f7e856de9b9da2b67dd32ab33e5"
server_key_iv: "a6f78293658831f20959b810d29f97ce"
server_key_version: "encryptionkey@0"
I20260806 18:34:33.448560 19736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:33.462836 19736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:33.466040 19736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:33.466256 19736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:33.466883 19736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:33.467775 19736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:33.467849 19736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:33.467926 19736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:33.467981 19736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:33.477962 19736 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:37695
I20260806 18:34:33.477991 19868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:37695 every 8 connection(s)
I20260806 18:34:33.478945 19736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:33.488320 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 19736
I20260806 18:34:33.488473 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:33.488816 18934 external_mini_cluster.cc:1428] Setting key 4e483554af47c393f0014df7188119cf
Aug 06 18:34:33 dist-test-slave-8w9v krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041273, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:33.495362 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.480986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:50867 (local address 127.18.125.190:36725)
0806 18:34:33.481180 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:33.481185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:33.482087 (+ 902us) server_negotiation.cc:407] Connection header received
0806 18:34:33.483035 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:33.483038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:33.483089 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:33.483169 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:33.484565 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.485235 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.486559 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.486791 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.486889 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.490293 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:33.490303 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:33.490305 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:33.490333 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:33.492055 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.492764 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.492770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.492772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.492871 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.493318 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.493323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.493325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.493551 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:33.493679 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:33.494855 (+ 1176us) server_negotiation.cc:299] Negotiation successful
0806 18:34:33.495115 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 18:34:33.496893 19871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.481406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:36725 (local address 127.18.125.130:50867)
0806 18:34:33.481917 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:33.481963 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:33.482815 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:33.483343 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:33.483350 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:33.483755 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:33.484381 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.484401 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.485396 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.485400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.486357 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.486375 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.486955 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.486958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.487120 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.487883 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:33.487912 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:33.490020 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:33.492289 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:33.492297 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:33.492315 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:33.492629 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:33.493024 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:33.493027 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:33.493030 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:33.493205 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:33.493679 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:33.493688 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:33.494275 (+ 587us) client_negotiation.cc:769] Sending connection context
0806 18:34:33.494759 (+ 484us) client_negotiation.cc:240] Negotiation successful
0806 18:34:33.495297 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":118,"threads_started":1}
I20260806 18:34:33.498769 19869 heartbeater.cc:347] Connected to a master server at 127.18.125.190:36725
I20260806 18:34:33.499121 19869 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:34:33.499818 19869 heartbeater.cc:511] Master 127.18.125.190:36725 requested a full tablet report, sending...
I20260806 18:34:33.501230 19532 ts_manager.cc:194] Registered new tserver with Master: 2e0313f23110469b91db77bb91b6630c (127.18.125.130:37695)
I20260806 18:34:33.501953 19532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:50867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:34:33.523394 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36725
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:33.643692 19874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:33.644006 19874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:33.644057 19874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:33.648413 19874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:33.648487 19874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:33.648582 19874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:34:33.654296 19874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:33.655486 19874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:33.656761 19874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:33.667734 19879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:33.667712 19880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:33.667768 19874 server_base.cc:1034] running on GCE node
W20260806 18:34:33.667712 19882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:33.668612 19874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:33.669283 19874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:33.670526 19874 hybrid_clock.cc:648] HybridClock initialized: now 1786041273670493 us; error 44 us; skew 500 ppm
Aug 06 18:34:33 dist-test-slave-8w9v krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041273, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:33.674283 19874 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:34:33.675669 19874 webserver.cc:460] Webserver started at http://127.18.125.131:45713/ using document root <none> and password file <none>
I20260806 18:34:33.676677 19874 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:33.676816 19874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:33.677282 19874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:33.680281 19874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "911cc98c45dd4fd1a51db5a9c614c2b7"
format_stamp: "Formatted at 2026-08-06 18:34:33 on dist-test-slave-8w9v"
server_key: "2e9c680d4b0cb87e0ddaa46df43e4cb9"
server_key_iv: "727fed7e5b971f28c02e57e3e7c8cd0f"
server_key_version: "encryptionkey@0"
I20260806 18:34:33.680981 19874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "911cc98c45dd4fd1a51db5a9c614c2b7"
format_stamp: "Formatted at 2026-08-06 18:34:33 on dist-test-slave-8w9v"
server_key: "2e9c680d4b0cb87e0ddaa46df43e4cb9"
server_key_iv: "727fed7e5b971f28c02e57e3e7c8cd0f"
server_key_version: "encryptionkey@0"
I20260806 18:34:33.685158 19874 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:34:33.687776 19890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:33.689170 19874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:34:33.689313 19874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "911cc98c45dd4fd1a51db5a9c614c2b7"
format_stamp: "Formatted at 2026-08-06 18:34:33 on dist-test-slave-8w9v"
server_key: "2e9c680d4b0cb87e0ddaa46df43e4cb9"
server_key_iv: "727fed7e5b971f28c02e57e3e7c8cd0f"
server_key_version: "encryptionkey@0"
I20260806 18:34:33.689446 19874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:33.713712 19874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:33.717826 19874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:33.718079 19874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:33.718792 19874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:33.719769 19874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:33.719851 19874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:33.719929 19874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:33.719965 19874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:33.730376 19874 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:34999
I20260806 18:34:33.730398 20009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:34999 every 8 connection(s)
I20260806 18:34:33.731405 19874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:33.741194 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 19874
I20260806 18:34:33.741362 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:33.741660 18934 external_mini_cluster.cc:1428] Setting key 04b642276126925427f08e47de146693
Aug 06 18:34:33 dist-test-slave-8w9v krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041273, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:33.748374 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.734093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:59133 (local address 127.18.125.190:36725)
0806 18:34:33.734308 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:33.734313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:33.735500 (+ 1187us) server_negotiation.cc:407] Connection header received
0806 18:34:33.736583 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:33.736586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:33.736641 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:33.736725 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:33.738027 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.738574 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.739800 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.740051 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.740231 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.743509 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:33.743513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:33.743516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:33.743552 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:33.745408 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.746041 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.746044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.746046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.746137 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.746525 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.746529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.746531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.746747 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:33.746865 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:33.747710 (+ 845us) server_negotiation.cc:299] Negotiation successful
0806 18:34:33.747883 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 18:34:33.749936 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.734547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:36725 (local address 127.18.125.131:59133)
0806 18:34:33.735316 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:33.735362 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:33.736346 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:33.736919 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:33.736926 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:33.737322 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:33.737870 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.737886 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.738710 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.738712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.739653 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.739665 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.740307 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.740311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.740489 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.741226 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:33.741256 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:33.743267 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:33.745559 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:33.745567 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:33.745589 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:33.745917 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:33.746242 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:33.746245 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:33.746247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:33.746414 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:33.746867 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:33.746877 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:33.747197 (+ 320us) client_negotiation.cc:769] Sending connection context
0806 18:34:33.747668 (+ 471us) client_negotiation.cc:240] Negotiation successful
0806 18:34:33.748424 (+ 756us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":139,"threads_started":1}
I20260806 18:34:33.751776 20010 heartbeater.cc:347] Connected to a master server at 127.18.125.190:36725
I20260806 18:34:33.752216 20010 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:33.752913 20010 heartbeater.cc:511] Master 127.18.125.190:36725 requested a full tablet report, sending...
I20260806 18:34:33.754418 19532 ts_manager.cc:194] Registered new tserver with Master: 911cc98c45dd4fd1a51db5a9c614c2b7 (127.18.125.131:34999)
I20260806 18:34:33.755311 19532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:59133
I20260806 18:34:33.757722 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:33.772410 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.762724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.18.125.190:36725)
0806 18:34:33.762992 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:33.762996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:33.763239 (+ 243us) server_negotiation.cc:407] Connection header received
0806 18:34:33.763357 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:33.763360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:33.763550 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:33.763695 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:33.764947 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.765543 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.766776 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.767032 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.767134 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.768985 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:33.768988 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:33.768990 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:33.769017 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:33.770546 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.770974 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.770977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.770978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.771053 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.771329 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.771331 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.771332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.771568 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:33.771674 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:33.771901 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 18:34:33.772260 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260806 18:34:33.778174 19531 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:33.781442 19531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:33.797032 20022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.790525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:37695 (local address 127.0.0.1:38300)
0806 18:34:33.790915 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:33.790973 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:33.791102 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:33.791818 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:33.791827 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:33.791855 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:33.792032 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.792041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.793744 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.793747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.795109 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.795121 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.796415 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.796419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.796545 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.796585 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:34:33.796731 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 18:34:33.796875 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":61,"thread_start_us":106,"threads_started":1}
I20260806 18:34:33.797044 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.790660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.18.125.130:37695)
0806 18:34:33.791016 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:33.791026 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:33.791052 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:34:33.791304 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:33.791310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:33.791558 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:33.792240 (+ 682us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:33.792423 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.793614 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.795271 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.796280 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.796419 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.796572 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:33.796770 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 18:34:33.796875 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":145,"threads_started":1}
I20260806 18:34:33.797578 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.791687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:38171 (local address 127.0.0.1:55506)
0806 18:34:33.791893 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:33.791908 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:33.792025 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:33.793204 (+ 1179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:33.793208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:33.793251 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:33.793348 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.793353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.794715 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.794718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.795927 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.795939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.797044 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.797047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.797226 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.797250 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:34:33.797316 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:34:33.797423 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":53,"thread_start_us":88,"threads_started":1}
I20260806 18:34:33.797729 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.791339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:34999 (local address 127.0.0.1:59570)
0806 18:34:33.791602 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:33.791618 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:33.791699 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:33.792960 (+ 1261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:33.792963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:33.792982 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:33.793110 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.793121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.794616 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.794621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.795821 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.795832 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.797011 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.797018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.797454 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.797476 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:34:33.797538 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:34:33.797605 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":264,"thread_start_us":120,"threads_started":1}
I20260806 18:34:33.798278 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.792165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.18.125.129:38171)
0806 18:34:33.792722 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:33.792728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:33.792752 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:34:33.792833 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:33.792838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:33.793047 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:33.793211 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:33.793481 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.794596 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.796091 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.796909 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.797784 (+ 875us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.797952 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:33.798009 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 18:34:33.798103 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":152,"threads_started":1}
I20260806 18:34:33.799125 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.791645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.18.125.131:34999)
0806 18:34:33.792499 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:33.792511 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:33.792533 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:34:33.792607 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:33.792616 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:33.792809 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:33.792967 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:33.793252 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.794454 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.795971 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.796851 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.798008 (+ 1157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.798133 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:33.798179 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 18:34:33.798270 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":151,"threads_started":1}
I20260806 18:34:33.800752 19662 tablet_service.cc:1467] Processing CreateTablet for tablet 1e10b395178347d38349e365e6b902f3 (DEFAULT_TABLE table=test-table [id=2b59138863be450394e34a375fdf6ad5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:33.800659 19801 tablet_service.cc:1467] Processing CreateTablet for tablet 1e10b395178347d38349e365e6b902f3 (DEFAULT_TABLE table=test-table [id=2b59138863be450394e34a375fdf6ad5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:33.801847 19662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e10b395178347d38349e365e6b902f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:33.802212 19801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e10b395178347d38349e365e6b902f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:33.802171 19941 tablet_service.cc:1467] Processing CreateTablet for tablet 1e10b395178347d38349e365e6b902f3 (DEFAULT_TABLE table=test-table [id=2b59138863be450394e34a375fdf6ad5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:33.803694 19941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e10b395178347d38349e365e6b902f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:33.810719 20028 tablet_bootstrap.cc:492] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d: Bootstrap starting.
I20260806 18:34:33.811671 20030 tablet_bootstrap.cc:492] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7: Bootstrap starting.
I20260806 18:34:33.812547 20029 tablet_bootstrap.cc:492] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c: Bootstrap starting.
I20260806 18:34:33.813524 20028 tablet_bootstrap.cc:654] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:33.814049 20030 tablet_bootstrap.cc:654] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:33.814611 20028 log.cc:824] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:33.815311 20029 tablet_bootstrap.cc:654] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:33.815521 20030 log.cc:824] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:33.816279 20029 log.cc:824] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:33.818317 20028 tablet_bootstrap.cc:492] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d: No bootstrap required, opened a new log
I20260806 18:34:33.818369 20030 tablet_bootstrap.cc:492] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7: No bootstrap required, opened a new log
I20260806 18:34:33.818574 20028 ts_tablet_manager.cc:1397] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 18:34:33.818579 20030 ts_tablet_manager.cc:1397] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:34:33.819231 20029 tablet_bootstrap.cc:492] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c: No bootstrap required, opened a new log
I20260806 18:34:33.819444 20029 ts_tablet_manager.cc:1397] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260806 18:34:33.822486 20028 raft_consensus.cc:348] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0313f23110469b91db77bb91b6630c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37695 } } peers { permanent_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 34999 } } peers { permanent_uuid: "b4215b39f333494086178ed8b62f116d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38171 } }
I20260806 18:34:33.822746 20030 raft_consensus.cc:348] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0313f23110469b91db77bb91b6630c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37695 } } peers { permanent_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 34999 } } peers { permanent_uuid: "b4215b39f333494086178ed8b62f116d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38171 } }
I20260806 18:34:33.823040 20030 raft_consensus.cc:374] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:33.823041 20028 raft_consensus.cc:374] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:33.823096 20030 raft_consensus.cc:729] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 911cc98c45dd4fd1a51db5a9c614c2b7, State: Initialized, Role: FOLLOWER
I20260806 18:34:33.823112 20028 raft_consensus.cc:729] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4215b39f333494086178ed8b62f116d, State: Initialized, Role: FOLLOWER
I20260806 18:34:33.823434 20029 raft_consensus.cc:348] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0313f23110469b91db77bb91b6630c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37695 } } peers { permanent_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 34999 } } peers { permanent_uuid: "b4215b39f333494086178ed8b62f116d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38171 } }
I20260806 18:34:33.823691 20029 raft_consensus.cc:374] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:33.823625 20030 consensus_queue.cc:260] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0313f23110469b91db77bb91b6630c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37695 } } peers { permanent_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 34999 } } peers { permanent_uuid: "b4215b39f333494086178ed8b62f116d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38171 } }
I20260806 18:34:33.823626 20028 consensus_queue.cc:260] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0313f23110469b91db77bb91b6630c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37695 } } peers { permanent_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 34999 } } peers { permanent_uuid: "b4215b39f333494086178ed8b62f116d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38171 } }
I20260806 18:34:33.823738 20029 raft_consensus.cc:729] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e0313f23110469b91db77bb91b6630c, State: Initialized, Role: FOLLOWER
I20260806 18:34:33.824198 20029 consensus_queue.cc:260] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0313f23110469b91db77bb91b6630c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37695 } } peers { permanent_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 34999 } } peers { permanent_uuid: "b4215b39f333494086178ed8b62f116d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38171 } }
I20260806 18:34:33.824797 20010 heartbeater.cc:503] Master 127.18.125.190:36725 was elected leader, sending a full tablet report...
I20260806 18:34:33.824827 20028 ts_tablet_manager.cc:1428] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260806 18:34:33.825096 20029 ts_tablet_manager.cc:1428] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:34:33.825096 19731 heartbeater.cc:503] Master 127.18.125.190:36725 was elected leader, sending a full tablet report...
I20260806 18:34:33.825423 19869 heartbeater.cc:503] Master 127.18.125.190:36725 was elected leader, sending a full tablet report...
I20260806 18:34:33.826259 20030 ts_tablet_manager.cc:1428] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 18:34:33.910105 20034 raft_consensus.cc:482] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:33.910337 20034 raft_consensus.cc:504] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0313f23110469b91db77bb91b6630c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37695 } } peers { permanent_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 34999 } } peers { permanent_uuid: "b4215b39f333494086178ed8b62f116d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38171 } }
I20260806 18:34:33.911537 20034 leader_election.cc:288] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e0313f23110469b91db77bb91b6630c (127.18.125.130:37695), b4215b39f333494086178ed8b62f116d (127.18.125.129:38171)
I20260806 18:34:33.915473 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.911855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:37695 (local address 127.18.125.131:58497)
0806 18:34:33.912000 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:33.912018 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:33.912181 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:33.912528 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:33.912532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:33.912551 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:33.912704 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.912710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.913475 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.913479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.914277 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.914287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.914999 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.915002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.915110 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.915134 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:34:33.915233 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 18:34:33.915332 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:34:33.915473 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.911993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:58497 (local address 127.18.125.130:37695)
0806 18:34:33.912173 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:33.912177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:33.912202 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:34:33.912326 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:33.912332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:33.912394 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:33.912515 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:33.912826 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.913353 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.914422 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.914857 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.914998 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.915060 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:33.915250 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 18:34:33.915333 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:34:33.916296 19823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e10b395178347d38349e365e6b902f3" candidate_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e0313f23110469b91db77bb91b6630c" is_pre_election: true
I20260806 18:34:33.916465 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.912552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:50115 (local address 127.18.125.129:38171)
0806 18:34:33.912779 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:33.912784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:33.912928 (+ 144us) server_negotiation.cc:407] Connection header received
0806 18:34:33.913086 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:33.913089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:33.913140 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:33.913229 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:33.913534 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.914036 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.915097 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.915662 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.915790 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.915836 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:33.916236 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 18:34:33.916338 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260806 18:34:33.916685 19823 raft_consensus.cc:2454] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 911cc98c45dd4fd1a51db5a9c614c2b7 in term 0.
I20260806 18:34:33.916877 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.912280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:38171 (local address 127.18.125.131:50115)
0806 18:34:33.912821 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:33.912842 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:33.912970 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:33.913245 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:33.913248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:33.913265 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:33.913413 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.913419 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.914161 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.914164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.914957 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:33.914967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.915923 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.915929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:33.916030 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.916049 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:34:33.916634 (+ 585us) client_negotiation.cc:240] Negotiation successful
0806 18:34:33.916714 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":265,"threads_started":1}
I20260806 18:34:33.917235 19893 leader_election.cc:302] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e0313f23110469b91db77bb91b6630c, 911cc98c45dd4fd1a51db5a9c614c2b7; no voters:
I20260806 18:34:33.917559 20034 raft_consensus.cc:2781] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:33.917677 20034 raft_consensus.cc:482] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:33.917747 20034 raft_consensus.cc:3037] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:33.918118 19684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e10b395178347d38349e365e6b902f3" candidate_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4215b39f333494086178ed8b62f116d" is_pre_election: true
I20260806 18:34:33.918421 19684 raft_consensus.cc:2454] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 911cc98c45dd4fd1a51db5a9c614c2b7 in term 0.
I20260806 18:34:33.919250 20034 raft_consensus.cc:504] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0313f23110469b91db77bb91b6630c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37695 } } peers { permanent_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 34999 } } peers { permanent_uuid: "b4215b39f333494086178ed8b62f116d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38171 } }
I20260806 18:34:33.919659 20034 leader_election.cc:288] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [CANDIDATE]: Term 1 election: Requested vote from peers 2e0313f23110469b91db77bb91b6630c (127.18.125.130:37695), b4215b39f333494086178ed8b62f116d (127.18.125.129:38171)
I20260806 18:34:33.920070 19684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e10b395178347d38349e365e6b902f3" candidate_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4215b39f333494086178ed8b62f116d"
I20260806 18:34:33.920109 19823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e10b395178347d38349e365e6b902f3" candidate_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e0313f23110469b91db77bb91b6630c"
I20260806 18:34:33.920265 19684 raft_consensus.cc:3037] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:33.920325 19823 raft_consensus.cc:3037] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:33.921408 19684 raft_consensus.cc:2454] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 911cc98c45dd4fd1a51db5a9c614c2b7 in term 1.
I20260806 18:34:33.921554 19823 raft_consensus.cc:2454] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 911cc98c45dd4fd1a51db5a9c614c2b7 in term 1.
I20260806 18:34:33.921779 19893 leader_election.cc:302] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 911cc98c45dd4fd1a51db5a9c614c2b7, b4215b39f333494086178ed8b62f116d; no voters:
I20260806 18:34:33.921981 20034 raft_consensus.cc:2781] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:33.922255 20034 raft_consensus.cc:686] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [term 1 LEADER]: Becoming Leader. State: Replica: 911cc98c45dd4fd1a51db5a9c614c2b7, State: Running, Role: LEADER
I20260806 18:34:33.922676 20034 consensus_queue.cc:237] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0313f23110469b91db77bb91b6630c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37695 } } peers { permanent_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 34999 } } peers { permanent_uuid: "b4215b39f333494086178ed8b62f116d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38171 } }
I20260806 18:34:33.926250 19531 catalog_manager.cc:5495] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 911cc98c45dd4fd1a51db5a9c614c2b7 (127.18.125.131). New cstate: current_term: 1 leader_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e0313f23110469b91db77bb91b6630c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "911cc98c45dd4fd1a51db5a9c614c2b7" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 34999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4215b39f333494086178ed8b62f116d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38171 } health_report { overall_health: UNKNOWN } } }
Aug 06 18:34:33 dist-test-slave-8w9v krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:34:33.959210 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.948072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.18.125.129:38171)
0806 18:34:33.948259 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:33.948290 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:33.948446 (+ 156us) server_negotiation.cc:407] Connection header received
0806 18:34:33.948625 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:33.948629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:33.948677 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:33.948772 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:33.949518 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.950047 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.950958 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.951185 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.951325 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.955159 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:33.955178 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:33.955195 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:33.955230 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:33.957130 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.957688 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.957695 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.957696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.957835 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.958147 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.958151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.958153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.958449 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:33.958688 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:33.958884 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 18:34:33.959026 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 18:34:33 dist-test-slave-8w9v krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041273, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260806 18:34:33.980304 19870 tablet.cc:2369] T 1e10b395178347d38349e365e6b902f3 P 2e0313f23110469b91db77bb91b6630c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:34:33.982880 20011 tablet.cc:2369] T 1e10b395178347d38349e365e6b902f3 P 911cc98c45dd4fd1a51db5a9c614c2b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 18:34:33 dist-test-slave-8w9v krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041273, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:34:33.989522 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:33.979641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55514 (local address 127.18.125.129:38171)
0806 18:34:33.979810 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:33.979817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:33.980233 (+ 416us) server_negotiation.cc:407] Connection header received
0806 18:34:33.980296 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:33.980298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:33.980347 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:33.980479 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:33.981376 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.981936 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.982815 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:33.983115 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:33.983203 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:33.986106 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:33.986110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:33.986112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:33.986144 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:33.987517 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.987960 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.987963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.987965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.988050 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:33.988374 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:33.988377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:33.988378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:33.988647 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:33.988811 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:33.989132 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 18:34:33.989349 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 18:34:33.993762 19732 tablet.cc:2369] T 1e10b395178347d38349e365e6b902f3 P b4215b39f333494086178ed8b62f116d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 18:34:34 dist-test-slave-8w9v krb5kdc[19485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:34:34 dist-test-slave-8w9v krb5kdc[19485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:34:34.023329 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:34.011321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55528 (local address 127.18.125.129:38171)
0806 18:34:34.011547 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:34.011553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:34.011736 (+ 183us) server_negotiation.cc:407] Connection header received
0806 18:34:34.011926 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:34.011929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:34.012088 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:34.012284 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:34.013117 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:34.013664 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:34.014697 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:34.015316 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:34.015471 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:34.019072 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:34.019076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:34.019078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:34.019113 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:34.021381 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:34.021944 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:34.021948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:34.021949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:34.022072 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:34.022405 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:34.022409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:34.022410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:34.022621 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:34.022737 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:34.023060 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 18:34:34.023183 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 18:34:34.025069 19642 tablet_service.cc:2908] Rejecting scan request for tablet 1e10b395178347d38349e365e6b902f3: Uninitialized: clean time has not yet been initialized
I20260806 18:34:34.027818 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 19597
I20260806 18:34:34.035604 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 19736
I20260806 18:34:34.043273 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 19874
I20260806 18:34:34.051146 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 19499
2026-08-06T18:34:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2577 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:34 dist-test-slave-8w9v krb5kdc[20052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:34 dist-test-slave-8w9v krb5kdc[20052](info): set up 2 sockets
Aug 06 18:34:34 dist-test-slave-8w9v krb5kdc[20052](info): commencing operation
krb5kdc: starting...
W20260806 18:34:35.149853 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:34:35 dist-test-slave-8w9v krb5kdc[20052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:34:35.288816 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:38387
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:40493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:38387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:35.408262 20069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:35.408660 20069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:35.408757 20069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:35.413166 20069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:35.413291 20069 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:34:35.413326 20069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:35.413380 20069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:35.413441 20069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:35.413482 20069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:35.419329 20069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:40493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:38387
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:38387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:35.420634 20069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:35.421794 20069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:35.431500 20078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:35.431500 20075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:35.431578 20074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:35.431557 20069 server_base.cc:1034] running on GCE node
I20260806 18:34:35.432531 20069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:35.433805 20069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:35.435056 20069 hybrid_clock.cc:648] HybridClock initialized: now 1786041275435027 us; error 47 us; skew 500 ppm
Aug 06 18:34:35 dist-test-slave-8w9v krb5kdc[20052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:35.452113 20069 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:34:35.453625 20069 webserver.cc:460] Webserver started at http://127.18.125.190:39405/ using document root <none> and password file <none>
I20260806 18:34:35.454285 20069 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:35.454391 20069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:35.454643 20069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:35.457005 20069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "8a0f45b54f2b4ca5bf8dc141ce5179f6"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-8w9v"
server_key: "5f88df1445d74840ee1d2d9b00b64e6f"
server_key_iv: "b48ee7ec40b60fc5bbeda20ce0717c53"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.457626 20069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "8a0f45b54f2b4ca5bf8dc141ce5179f6"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-8w9v"
server_key: "5f88df1445d74840ee1d2d9b00b64e6f"
server_key_iv: "b48ee7ec40b60fc5bbeda20ce0717c53"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.461707 20069 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:34:35.464524 20086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:35.465852 20069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:34:35.466009 20069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "8a0f45b54f2b4ca5bf8dc141ce5179f6"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-8w9v"
server_key: "5f88df1445d74840ee1d2d9b00b64e6f"
server_key_iv: "b48ee7ec40b60fc5bbeda20ce0717c53"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.466138 20069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:35.487820 20069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:35.491387 20069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:35.491642 20069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:35.501645 20069 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:38387
I20260806 18:34:35.501813 20152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:38387 every 8 connection(s)
I20260806 18:34:35.502892 20069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:35.506188 20153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:35.506613 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 20069
I20260806 18:34:35.506738 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:35.507051 18934 external_mini_cluster.cc:1428] Setting key 75a2f53e6ffd626ac43707b12a9c6445
I20260806 18:34:35.514017 20153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6: Bootstrap starting.
Aug 06 18:34:35 dist-test-slave-8w9v krb5kdc[20052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:35.516963 20153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:35.517751 20153 log.cc:824] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:35.520488 20153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6: No bootstrap required, opened a new log
I20260806 18:34:35.523267 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.508692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42084 (local address 127.18.125.190:38387)
0806 18:34:35.509241 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:35.509257 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:35.509295 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:34:35.510231 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:35.510251 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:35.510611 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:35.510982 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:35.511825 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.512919 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.513818 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.514374 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.514553 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.517627 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:35.517632 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:35.517646 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:35.517676 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:35.520349 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.520877 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.520883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.520887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.521009 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.521312 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.521315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.521317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.521724 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:35.521947 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:35.522218 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 18:34:35.522523 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":174,"threads_started":1}
I20260806 18:34:35.524019 20153 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0f45b54f2b4ca5bf8dc141ce5179f6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38387 } }
I20260806 18:34:35.524349 20153 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:35.524421 20153 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a0f45b54f2b4ca5bf8dc141ce5179f6, State: Initialized, Role: FOLLOWER
I20260806 18:34:35.524863 20153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0f45b54f2b4ca5bf8dc141ce5179f6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38387 } }
I20260806 18:34:35.524999 20153 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:35.525097 20153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:35.525239 20153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:35.526508 20153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0f45b54f2b4ca5bf8dc141ce5179f6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38387 } }
I20260806 18:34:35.526880 20153 leader_election.cc:302] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a0f45b54f2b4ca5bf8dc141ce5179f6; no voters:
I20260806 18:34:35.527264 20153 leader_election.cc:288] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:35.527359 20158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:35.527549 20158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [term 1 LEADER]: Becoming Leader. State: Replica: 8a0f45b54f2b4ca5bf8dc141ce5179f6, State: Running, Role: LEADER
I20260806 18:34:35.527885 20158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0f45b54f2b4ca5bf8dc141ce5179f6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38387 } }
I20260806 18:34:35.528718 20153 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:35.529290 20160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a0f45b54f2b4ca5bf8dc141ce5179f6. Latest consensus state: current_term: 1 leader_uuid: "8a0f45b54f2b4ca5bf8dc141ce5179f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0f45b54f2b4ca5bf8dc141ce5179f6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38387 } } }
I20260806 18:34:35.529439 20160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:35.529711 20159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a0f45b54f2b4ca5bf8dc141ce5179f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0f45b54f2b4ca5bf8dc141ce5179f6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38387 } } }
I20260806 18:34:35.529820 20159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:35.529956 20163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:35.533099 20163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:35.539518 20163 catalog_manager.cc:1347] Generated new cluster ID: 2a869ed3f7fc43b5b5045702762a7213
I20260806 18:34:35.539606 20163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:35.570255 20163 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:35.571401 20163 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:35.582535 20163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6: Generated new TSK 0
I20260806 18:34:35.583307 20163 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:34:35.620512 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:38387
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:40493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:35.750952 20182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:35.751783 20182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:35.751899 20182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:35.758230 20182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:35.758319 20182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:35.758368 20182 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:35.758406 20182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:34:35.764554 20182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:40493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.125.190:38387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:35.765733 20182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:35.766786 20182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:35.778993 20190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:35.779093 20182 server_base.cc:1034] running on GCE node
W20260806 18:34:35.779142 20188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:35.779143 20187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:35.780030 20182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:35.780943 20182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:35.782187 20182 hybrid_clock.cc:648] HybridClock initialized: now 1786041275782135 us; error 79 us; skew 500 ppm
Aug 06 18:34:35 dist-test-slave-8w9v krb5kdc[20052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:35.785682 20182 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:34:35.786900 20182 webserver.cc:460] Webserver started at http://127.18.125.129:43389/ using document root <none> and password file <none>
I20260806 18:34:35.787531 20182 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:35.787619 20182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:35.787807 20182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:35.790185 20182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "d9939f1e14cd450f99cfda12d85c82ae"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-8w9v"
server_key: "9993ad4208449ad021cda2173281268b"
server_key_iv: "9aef20e2d41eb7204629c216ef373313"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.790764 20182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "d9939f1e14cd450f99cfda12d85c82ae"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-8w9v"
server_key: "9993ad4208449ad021cda2173281268b"
server_key_iv: "9aef20e2d41eb7204629c216ef373313"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.796903 20182 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 18:34:35.799978 20198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:35.801710 20182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:34:35.801890 20182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "d9939f1e14cd450f99cfda12d85c82ae"
format_stamp: "Formatted at 2026-08-06 18:34:35 on dist-test-slave-8w9v"
server_key: "9993ad4208449ad021cda2173281268b"
server_key_iv: "9aef20e2d41eb7204629c216ef373313"
server_key_version: "encryptionkey@0"
I20260806 18:34:35.802042 20182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:35.817909 20182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:35.821609 20182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:35.821861 20182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:35.823751 20182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:35.823839 20182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:35.823907 20182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:35.823944 20182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:35 dist-test-slave-8w9v krb5kdc[20052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:35.835995 20182 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:44275
I20260806 18:34:35.836084 20325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:44275 every 8 connection(s)
I20260806 18:34:35.837071 20182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:35.839293 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.824834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:35815 (local address 127.18.125.190:38387)
0806 18:34:35.825042 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:35.825048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:35.825983 (+ 935us) server_negotiation.cc:407] Connection header received
0806 18:34:35.827079 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:35.827085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:35.827157 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:35.827276 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:35.828842 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.829547 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.830988 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.831303 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.831440 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.835128 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:35.835133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:35.835137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:35.835174 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:35.836875 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.837544 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.837566 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.837568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.837670 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.838137 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.838140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.838142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.838428 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:35.838536 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:35.839048 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 18:34:35.839167 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 18:34:35.839681 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 20182
I20260806 18:34:35.839804 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:35.840104 18934 external_mini_cluster.cc:1428] Setting key b3b98768226eb0fa0be7883d18ab0ca1
I20260806 18:34:35.840332 20208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.825291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:38387 (local address 127.18.125.129:35815)
0806 18:34:35.825789 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:35.825841 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:35.826784 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:35.827469 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:35.827475 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:35.827932 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:35.828686 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:35.828703 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.829691 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.829698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:35.830851 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:35.830862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.831529 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.831535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:35.831910 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.832998 (+ 1088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:35.833024 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:35.834932 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:35.837015 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:35.837022 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:35.837036 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:35.837431 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:35.837794 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:35.837799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:35.837802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:35.838037 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:35.838562 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:35.838569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:35.838817 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 18:34:35.838988 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 18:34:35.839253 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":142,"threads_started":1}
I20260806 18:34:35.841832 20326 heartbeater.cc:347] Connected to a master server at 127.18.125.190:38387
I20260806 18:34:35.842025 20326 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:35.842571 20326 heartbeater.cc:511] Master 127.18.125.190:38387 requested a full tablet report, sending...
I20260806 18:34:35.844507 20103 ts_manager.cc:194] Registered new tserver with Master: d9939f1e14cd450f99cfda12d85c82ae (127.18.125.129:44275)
I20260806 18:34:35.845845 20103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:35815
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:34:35.853432 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.844866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:56907 (local address 127.18.125.190:38387)
0806 18:34:35.844999 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:35.845003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:35.846710 (+ 1707us) server_negotiation.cc:407] Connection header received
0806 18:34:35.846845 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:35.846850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:35.846903 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:35.847003 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:35.847445 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.847971 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.848878 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.849121 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.849239 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.850470 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:35.850474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:35.850477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:35.850502 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:35.851909 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.852378 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.852381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.852382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.852461 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:35.852691 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:35.852693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:35.852694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:35.852856 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:35.852930 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:35.853148 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 18:34:35.853299 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:34:35.853451 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:35.844553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:38387 (local address 127.18.125.129:56907)
0806 18:34:35.846603 (+ 2050us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:35.846620 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:35.846728 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:35.847008 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:35.847010 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:35.847199 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:35.847315 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:35.847321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.848110 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.848114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:35.848746 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:35.848755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:35.849247 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:35.849251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:35.849363 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:35.849948 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:35.849960 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:35.850314 (+ 354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:35.852028 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:35.852031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:35.852033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:35.852298 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:35.852550 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:35.852552 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:35.852553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:35.852632 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:35.852946 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:35.852948 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:35.853027 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 18:34:35.853108 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:34:35.853264 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1954,"thread_start_us":130,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:34:35.871515 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:38387
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:40493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:35.994239 20335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:35.994599 20335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:35.994683 20335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:35.999436 20335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:35.999537 20335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:35.999614 20335 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:35.999706 20335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:34:36.005646 20335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:40493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.125.190:38387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:36.007014 20335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:36.008244 20335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:36.020512 20342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:36.020529 20341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:36.020591 20335 server_base.cc:1034] running on GCE node
W20260806 18:34:36.020772 20344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:36.021243 20335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:36.022029 20335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:36.025431 20335 hybrid_clock.cc:648] HybridClock initialized: now 1786041276025370 us; error 81 us; skew 500 ppm
Aug 06 18:34:36 dist-test-slave-8w9v krb5kdc[20052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:36.028939 20335 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:34:36.030177 20335 webserver.cc:460] Webserver started at http://127.18.125.130:33271/ using document root <none> and password file <none>
I20260806 18:34:36.030838 20335 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:36.030953 20335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:36.031217 20335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:36.033746 20335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "6df850a8c0de4df4ba6eac63469c075e"
format_stamp: "Formatted at 2026-08-06 18:34:36 on dist-test-slave-8w9v"
server_key: "22537fc4ab2ae5a2d0e87885643b3e0d"
server_key_iv: "7fdc4d43fbb80742793da13bdc92c78b"
server_key_version: "encryptionkey@0"
I20260806 18:34:36.034407 20335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "6df850a8c0de4df4ba6eac63469c075e"
format_stamp: "Formatted at 2026-08-06 18:34:36 on dist-test-slave-8w9v"
server_key: "22537fc4ab2ae5a2d0e87885643b3e0d"
server_key_iv: "7fdc4d43fbb80742793da13bdc92c78b"
server_key_version: "encryptionkey@0"
I20260806 18:34:36.038767 20335 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:36.041841 20351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:36.043644 20335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:34:36.043817 20335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "6df850a8c0de4df4ba6eac63469c075e"
format_stamp: "Formatted at 2026-08-06 18:34:36 on dist-test-slave-8w9v"
server_key: "22537fc4ab2ae5a2d0e87885643b3e0d"
server_key_iv: "7fdc4d43fbb80742793da13bdc92c78b"
server_key_version: "encryptionkey@0"
I20260806 18:34:36.044003 20335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:36.075414 20335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:36.079329 20335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:36.079586 20335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:36.081357 20335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:36.081470 20335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:36.081557 20335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:36.081584 20335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:36 dist-test-slave-8w9v krb5kdc[20052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:36.094121 20335 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:38457
I20260806 18:34:36.094225 20473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:38457 every 8 connection(s)
I20260806 18:34:36.095136 20335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:36.096930 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.082412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:34939 (local address 127.18.125.190:38387)
0806 18:34:36.082556 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.082561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.083643 (+ 1082us) server_negotiation.cc:407] Connection header received
0806 18:34:36.084844 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.084851 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.084920 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.085012 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:36.086493 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.087225 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.088616 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.088961 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.089078 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.092786 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:36.092790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:36.092792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:36.092823 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:36.094552 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.095126 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.095130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.095131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.095243 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.095626 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.095630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.095631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.095845 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:36.095972 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.096605 (+ 633us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.096740 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:34:36.097728 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.082911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:38387 (local address 127.18.125.130:34939)
0806 18:34:36.083442 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.083500 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.084552 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.085460 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.085467 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.085826 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:36.086327 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.086342 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.087375 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.087379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.088467 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.088479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.089121 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.089125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.089310 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.090059 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:36.090104 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:36.092581 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:36.094688 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:36.094694 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:36.094710 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:36.095001 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:36.095352 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:36.095355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:36.095357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:36.095514 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:36.095972 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:36.095980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:36.096331 (+ 351us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.096545 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.096886 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":140,"threads_started":1}
I20260806 18:34:36.099256 20474 heartbeater.cc:347] Connected to a master server at 127.18.125.190:38387
I20260806 18:34:36.099467 20474 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:36.099944 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 20335
I20260806 18:34:36.100057 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:36.100098 20474 heartbeater.cc:511] Master 127.18.125.190:38387 requested a full tablet report, sending...
I20260806 18:34:36.100433 18934 external_mini_cluster.cc:1428] Setting key 087955ee8100cf88fac252af4e111427
I20260806 18:34:36.101498 20101 ts_manager.cc:194] Registered new tserver with Master: 6df850a8c0de4df4ba6eac63469c075e (127.18.125.130:38457)
I20260806 18:34:36.102401 20101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:34939
I20260806 18:34:36.111912 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.102280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:50849 (local address 127.18.125.190:38387)
0806 18:34:36.102437 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.102443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.102915 (+ 472us) server_negotiation.cc:407] Connection header received
0806 18:34:36.103112 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.103116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.103173 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.103285 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:36.103954 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.104777 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.106166 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.106429 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.106578 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.108089 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:36.108092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:36.108095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:36.108190 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:36.109842 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.110357 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.110366 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.110367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.110456 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.110825 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.110828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.110830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.111142 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:36.111235 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.111667 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.111780 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:34:36.112089 20480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.102430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:38387 (local address 127.18.125.130:50849)
0806 18:34:36.102819 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.102839 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.103000 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.103327 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.103333 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.103606 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:36.103787 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.103798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.104951 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.104956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.106022 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.106039 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.106602 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.106609 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.106752 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.107420 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:36.107435 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:36.107924 (+ 489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:36.109990 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:36.109994 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:36.109997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:36.110236 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:36.110562 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:36.110565 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:36.110567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:36.110660 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:36.111295 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:36.111299 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:36.111433 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.111600 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.111813 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":145,"threads_started":1}
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:34:36.133410 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:38387
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:40493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:36.255573 20483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:36.255894 20483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:36.255946 20483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:36.260375 20483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:36.260453 20483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:36.260499 20483 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:36.260536 20483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:34:36.266345 20483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:40493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.125.190:38387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:36.267524 20483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:36.268643 20483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:36.279639 20488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:36.279738 20483 server_base.cc:1034] running on GCE node
W20260806 18:34:36.279680 20489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:36.279676 20491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:36.280498 20483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:36.281198 20483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:36.282364 20483 hybrid_clock.cc:648] HybridClock initialized: now 1786041276282335 us; error 43 us; skew 500 ppm
Aug 06 18:34:36 dist-test-slave-8w9v krb5kdc[20052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:36.285524 20483 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:34:36.286471 20483 webserver.cc:460] Webserver started at http://127.18.125.131:39885/ using document root <none> and password file <none>
I20260806 18:34:36.287081 20483 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:36.287145 20483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:36.287412 20483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:36.289700 20483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "87c334f16375472b8a002c6a2f6bae3a"
format_stamp: "Formatted at 2026-08-06 18:34:36 on dist-test-slave-8w9v"
server_key: "81c793d48260d2db33b731674f5f83e4"
server_key_iv: "4385fdf27a5c1e1ae1db2739ce7c1f1f"
server_key_version: "encryptionkey@0"
I20260806 18:34:36.290304 20483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "87c334f16375472b8a002c6a2f6bae3a"
format_stamp: "Formatted at 2026-08-06 18:34:36 on dist-test-slave-8w9v"
server_key: "81c793d48260d2db33b731674f5f83e4"
server_key_iv: "4385fdf27a5c1e1ae1db2739ce7c1f1f"
server_key_version: "encryptionkey@0"
I20260806 18:34:36.294435 20483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:36.297086 20499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:36.298341 20483 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:34:36.298491 20483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "87c334f16375472b8a002c6a2f6bae3a"
format_stamp: "Formatted at 2026-08-06 18:34:36 on dist-test-slave-8w9v"
server_key: "81c793d48260d2db33b731674f5f83e4"
server_key_iv: "4385fdf27a5c1e1ae1db2739ce7c1f1f"
server_key_version: "encryptionkey@0"
I20260806 18:34:36.298601 20483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:36.315096 20483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:36.318615 20483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:36.318840 20483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:36.320454 20483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:36.320505 20483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:36.320571 20483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:36.320603 20483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:36 dist-test-slave-8w9v krb5kdc[20052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:36.332437 20483 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:39679
I20260806 18:34:36.332500 20619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:39679 every 8 connection(s)
I20260806 18:34:36.333452 20483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:36.337345 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.321424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:41919 (local address 127.18.125.190:38387)
0806 18:34:36.321608 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.321613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.322520 (+ 907us) server_negotiation.cc:407] Connection header received
0806 18:34:36.323858 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.323865 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.323938 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.324006 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:36.326058 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.326700 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.328701 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.329080 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.329190 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.332859 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:36.332864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:36.332866 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:36.332904 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:36.334682 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.335384 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.335388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.335390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.335488 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.335915 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.335918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.335920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.336260 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:36.336394 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.336994 (+ 600us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.337147 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:34:36.338353 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.321812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:38387 (local address 127.18.125.131:41919)
0806 18:34:36.322321 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.322381 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.323570 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.324529 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.324540 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.325048 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:36.325850 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.325874 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.326850 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.326856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.328531 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.328548 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.329272 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.329276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.329526 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.330513 (+ 987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:36.330542 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:36.332612 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:36.334825 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:36.334834 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:36.334852 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:36.335246 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:36.335614 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:36.335618 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:36.335621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:36.335789 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:36.336394 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:36.336404 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:36.336703 (+ 299us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.336924 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.337312 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":110,"threads_started":1}
I20260806 18:34:36.339814 20620 heartbeater.cc:347] Connected to a master server at 127.18.125.190:38387
I20260806 18:34:36.340039 20620 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:36.340723 20620 heartbeater.cc:511] Master 127.18.125.190:38387 requested a full tablet report, sending...
I20260806 18:34:36.341786 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 20483
I20260806 18:34:36.341924 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:36.342135 20103 ts_manager.cc:194] Registered new tserver with Master: 87c334f16375472b8a002c6a2f6bae3a (127.18.125.131:39679)
I20260806 18:34:36.342238 18934 external_mini_cluster.cc:1428] Setting key abedb9fea84af8f1199d1b4d6575a9ce
I20260806 18:34:36.343066 20103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:41919
I20260806 18:34:36.344712 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:36.352587 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.343838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:46003 (local address 127.18.125.190:38387)
0806 18:34:36.343961 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.343966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.344427 (+ 461us) server_negotiation.cc:407] Connection header received
0806 18:34:36.344621 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.344627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.344680 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.344789 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:36.345344 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.345869 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.346860 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.347092 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.347199 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.348736 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:36.348739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:36.348741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:36.348765 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:36.350331 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.350868 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.350873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.350874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.350972 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.351313 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.351317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.351319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.351605 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:36.351717 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.352191 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.352350 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:34:36.352706 20626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.343672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:38387 (local address 127.18.125.131:46003)
0806 18:34:36.344334 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.344351 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.344503 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.344814 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.344817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.345102 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:36.345216 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.345222 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.346007 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.346010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.346732 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.346742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.347216 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.347220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.347327 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.348025 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:36.348046 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:36.348576 (+ 530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:36.350459 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:36.350464 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:36.350466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:36.350749 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:36.351116 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:36.351118 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:36.351120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:36.351207 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:36.351717 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:36.351720 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:36.351810 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.351903 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.352394 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":135,"threads_started":1}
Aug 06 18:34:36 dist-test-slave-8w9v krb5kdc[20052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:34:36 dist-test-slave-8w9v krb5kdc[20052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:36.375149 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.366000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42096 (local address 127.18.125.190:38387)
0806 18:34:36.366164 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.366167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.366182 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:34:36.366226 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.366228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.366270 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.366352 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:36.367138 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.367692 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.368652 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.368901 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.369096 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.371841 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:36.371845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:36.371847 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:36.371879 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:36.373396 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.373891 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.373894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.373903 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.373983 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.374297 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.374299 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.374300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.374490 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:36.374592 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.374859 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.375020 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 18:34:36.378535 20103 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:36.382023 20103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:36.398793 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.392832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.18.125.130:38457)
0806 18:34:36.393304 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.393316 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.393478 (+ 162us) server_negotiation.cc:407] Connection header received
0806 18:34:36.393680 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.393687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.393909 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.394264 (+ 355us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:36.394532 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.395666 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.397405 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.398037 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.398205 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.398378 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.398512 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.398620 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":155,"threads_started":1}
I20260806 18:34:36.398782 20641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.392341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:38457 (local address 127.0.0.1:36088)
0806 18:34:36.393325 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.393344 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.393509 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.394250 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.394252 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.394273 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:36.394395 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.394401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.395802 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.395806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.397275 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.397283 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.398187 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.398193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.398318 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.398360 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.398483 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.398620 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"mutex_wait_us":132,"thread_start_us":119,"threads_started":1}
I20260806 18:34:36.399456 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.391877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:39679 (local address 127.0.0.1:57558)
0806 18:34:36.392520 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.392575 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.392721 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.393586 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.393593 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.393620 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:36.393776 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.393784 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.395474 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.395478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.397125 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.397137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.398629 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.398634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.398793 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.399197 (+ 404us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.399271 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.399331 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":147,"thread_start_us":96,"threads_started":1}
I20260806 18:34:36.399857 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.392989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54656 (local address 127.18.125.129:44275)
0806 18:34:36.393725 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.393734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.394211 (+ 477us) server_negotiation.cc:407] Connection header received
0806 18:34:36.394297 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.394302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.394499 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.394665 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:36.394908 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.395828 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.397349 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.398244 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.398391 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.398540 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.399534 (+ 994us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.399727 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":76,"threads_started":1}
I20260806 18:34:36.400059 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.392060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57558 (local address 127.18.125.131:39679)
0806 18:34:36.392594 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.392611 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.392641 (+ 30us) server_negotiation.cc:407] Connection header received
0806 18:34:36.392976 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.392983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.393221 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.393409 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:36.393938 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.395318 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.397281 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.398243 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.398423 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.398547 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.399671 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.399799 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":192,"threads_started":1}
I20260806 18:34:36.399456 20640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.392739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44275 (local address 127.0.0.1:54656)
0806 18:34:36.393866 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.393880 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.393973 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.394649 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.394652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.394671 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:36.394785 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.394791 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.395971 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.395974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.397158 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.397170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.398629 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.398634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.398840 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.398859 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.399144 (+ 285us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.399295 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1064,"mutex_wait_us":50,"thread_start_us":128,"threads_started":1}
I20260806 18:34:36.403061 20405 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab68291a7f341b7a6c5ba7aef6c5803 (DEFAULT_TABLE table=test-table [id=c2f412a8b77b4b95895acaffdae6a98c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:36.403854 20551 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab68291a7f341b7a6c5ba7aef6c5803 (DEFAULT_TABLE table=test-table [id=c2f412a8b77b4b95895acaffdae6a98c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:36.404426 20405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ab68291a7f341b7a6c5ba7aef6c5803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:36.404968 20551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ab68291a7f341b7a6c5ba7aef6c5803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:36.406581 20253 tablet_service.cc:1467] Processing CreateTablet for tablet 6ab68291a7f341b7a6c5ba7aef6c5803 (DEFAULT_TABLE table=test-table [id=c2f412a8b77b4b95895acaffdae6a98c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:36.407753 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ab68291a7f341b7a6c5ba7aef6c5803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:36.412874 20644 tablet_bootstrap.cc:492] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e: Bootstrap starting.
I20260806 18:34:36.414539 20645 tablet_bootstrap.cc:492] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a: Bootstrap starting.
I20260806 18:34:36.415390 20646 tablet_bootstrap.cc:492] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae: Bootstrap starting.
I20260806 18:34:36.416563 20644 tablet_bootstrap.cc:654] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:36.416818 20645 tablet_bootstrap.cc:654] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:36.417585 20644 log.cc:824] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:36.417585 20645 log.cc:824] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:36.418025 20646 tablet_bootstrap.cc:654] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:36.418872 20646 log.cc:824] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:36.419904 20644 tablet_bootstrap.cc:492] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e: No bootstrap required, opened a new log
I20260806 18:34:36.420435 20644 ts_tablet_manager.cc:1397] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260806 18:34:36.420677 20646 tablet_bootstrap.cc:492] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae: No bootstrap required, opened a new log
I20260806 18:34:36.420670 20645 tablet_bootstrap.cc:492] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a: No bootstrap required, opened a new log
I20260806 18:34:36.420866 20645 ts_tablet_manager.cc:1397] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 18:34:36.420903 20646 ts_tablet_manager.cc:1397] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:34:36.424950 20646 raft_consensus.cc:348] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } }
I20260806 18:34:36.425345 20646 raft_consensus.cc:374] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:36.425424 20646 raft_consensus.cc:729] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9939f1e14cd450f99cfda12d85c82ae, State: Initialized, Role: FOLLOWER
I20260806 18:34:36.425227 20645 raft_consensus.cc:348] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } }
I20260806 18:34:36.425227 20644 raft_consensus.cc:348] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } }
I20260806 18:34:36.425518 20644 raft_consensus.cc:374] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:36.425518 20645 raft_consensus.cc:374] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:36.425581 20645 raft_consensus.cc:729] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87c334f16375472b8a002c6a2f6bae3a, State: Initialized, Role: FOLLOWER
I20260806 18:34:36.425581 20644 raft_consensus.cc:729] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df850a8c0de4df4ba6eac63469c075e, State: Initialized, Role: FOLLOWER
I20260806 18:34:36.425989 20646 consensus_queue.cc:260] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } }
I20260806 18:34:36.426054 20645 consensus_queue.cc:260] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } }
I20260806 18:34:36.426046 20644 consensus_queue.cc:260] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } }
I20260806 18:34:36.426798 20326 heartbeater.cc:503] Master 127.18.125.190:38387 was elected leader, sending a full tablet report...
I20260806 18:34:36.427161 20646 ts_tablet_manager.cc:1428] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:34:36.427174 20645 ts_tablet_manager.cc:1428] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:34:36.427582 20620 heartbeater.cc:503] Master 127.18.125.190:38387 was elected leader, sending a full tablet report...
I20260806 18:34:36.428042 20474 heartbeater.cc:503] Master 127.18.125.190:38387 was elected leader, sending a full tablet report...
I20260806 18:34:36.429528 20644 ts_tablet_manager.cc:1428] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e: Time spent starting tablet: real 0.009s user 0.002s sys 0.005s
I20260806 18:34:36.482355 20650 raft_consensus.cc:482] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:36.482631 20650 raft_consensus.cc:504] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } }
I20260806 18:34:36.483924 20650 leader_election.cc:288] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87c334f16375472b8a002c6a2f6bae3a (127.18.125.131:39679), 6df850a8c0de4df4ba6eac63469c075e (127.18.125.130:38457)
I20260806 18:34:36.488308 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.484562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:40963 (local address 127.18.125.130:38457)
0806 18:34:36.484693 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.484698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.484870 (+ 172us) server_negotiation.cc:407] Connection header received
0806 18:34:36.485031 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.485034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.485086 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.485172 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:36.485468 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.486005 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.487144 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.487630 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.487771 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.487848 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.488060 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.488161 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:34:36.488322 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.484411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:60517 (local address 127.18.125.131:39679)
0806 18:34:36.484575 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.484580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.484922 (+ 342us) server_negotiation.cc:407] Connection header received
0806 18:34:36.485094 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.485097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.485141 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.485224 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:36.485534 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.486068 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.487174 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.487609 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.487778 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.487849 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.488076 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.488193 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 18:34:36.488358 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.484296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:39679 (local address 127.18.125.129:60517)
0806 18:34:36.484844 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.484857 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.484951 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.485250 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.485278 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.485295 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:36.485414 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.485419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.486188 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.486195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.487041 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.487051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.487764 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.487768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.487943 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.487962 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.488056 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.488152 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":201,"thread_start_us":89,"threads_started":1}
I20260806 18:34:36.488363 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.484299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:38457 (local address 127.18.125.129:40963)
0806 18:34:36.484755 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.484775 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.484924 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.485185 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.485188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.485205 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:36.485347 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.485353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.486134 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.486140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.487005 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.487015 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.487764 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.487768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.487919 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.487949 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.488042 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.488124 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":22,"thread_start_us":147,"threads_started":1}
I20260806 18:34:36.489204 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab68291a7f341b7a6c5ba7aef6c5803" candidate_uuid: "d9939f1e14cd450f99cfda12d85c82ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c334f16375472b8a002c6a2f6bae3a" is_pre_election: true
I20260806 18:34:36.489187 20425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab68291a7f341b7a6c5ba7aef6c5803" candidate_uuid: "d9939f1e14cd450f99cfda12d85c82ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6df850a8c0de4df4ba6eac63469c075e" is_pre_election: true
I20260806 18:34:36.489552 20572 raft_consensus.cc:2454] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9939f1e14cd450f99cfda12d85c82ae in term 0.
I20260806 18:34:36.489552 20425 raft_consensus.cc:2454] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9939f1e14cd450f99cfda12d85c82ae in term 0.
I20260806 18:34:36.490110 20200 leader_election.cc:302] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6df850a8c0de4df4ba6eac63469c075e, 87c334f16375472b8a002c6a2f6bae3a, d9939f1e14cd450f99cfda12d85c82ae; no voters:
I20260806 18:34:36.490435 20650 raft_consensus.cc:2781] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:36.490515 20650 raft_consensus.cc:482] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:36.490545 20650 raft_consensus.cc:3037] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:36.491716 20650 raft_consensus.cc:504] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } }
I20260806 18:34:36.492185 20650 leader_election.cc:288] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [CANDIDATE]: Term 1 election: Requested vote from peers 87c334f16375472b8a002c6a2f6bae3a (127.18.125.131:39679), 6df850a8c0de4df4ba6eac63469c075e (127.18.125.130:38457)
I20260806 18:34:36.492559 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab68291a7f341b7a6c5ba7aef6c5803" candidate_uuid: "d9939f1e14cd450f99cfda12d85c82ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c334f16375472b8a002c6a2f6bae3a"
I20260806 18:34:36.492632 20425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ab68291a7f341b7a6c5ba7aef6c5803" candidate_uuid: "d9939f1e14cd450f99cfda12d85c82ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6df850a8c0de4df4ba6eac63469c075e"
I20260806 18:34:36.492714 20572 raft_consensus.cc:3037] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:36.492751 20425 raft_consensus.cc:3037] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:36.494233 20572 raft_consensus.cc:2454] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9939f1e14cd450f99cfda12d85c82ae in term 1.
I20260806 18:34:36.494603 20200 leader_election.cc:302] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87c334f16375472b8a002c6a2f6bae3a, d9939f1e14cd450f99cfda12d85c82ae; no voters:
I20260806 18:34:36.494630 20425 raft_consensus.cc:2454] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9939f1e14cd450f99cfda12d85c82ae in term 1.
I20260806 18:34:36.494863 20650 raft_consensus.cc:2781] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:36.495123 20650 raft_consensus.cc:686] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 1 LEADER]: Becoming Leader. State: Replica: d9939f1e14cd450f99cfda12d85c82ae, State: Running, Role: LEADER
I20260806 18:34:36.495529 20650 consensus_queue.cc:237] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } }
I20260806 18:34:36.499471 20103 catalog_manager.cc:5495] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae reported cstate change: term changed from 0 to 1, leader changed from <none> to d9939f1e14cd450f99cfda12d85c82ae (127.18.125.129). New cstate: current_term: 1 leader_uuid: "d9939f1e14cd450f99cfda12d85c82ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:36.530754 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.526758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.18.125.129:44275)
0806 18:34:36.526902 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.526906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.526924 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:34:36.527070 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.527072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.527131 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.527228 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:36.527508 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.528025 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.529114 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.529332 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.529502 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.529878 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:36.530312 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:36.530405 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.530561 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.530633 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:34:36.537252 20425 raft_consensus.cc:1261] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e [term 1 FOLLOWER]: Refusing update from remote peer d9939f1e14cd450f99cfda12d85c82ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:36.537498 20572 raft_consensus.cc:1261] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a [term 1 FOLLOWER]: Refusing update from remote peer d9939f1e14cd450f99cfda12d85c82ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:36.538157 20655 consensus_queue.cc:1035] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:36.538373 20650 consensus_queue.cc:1035] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:36.546420 20659 mvcc.cc:205] Tried to move back new op lower bound from 7315625068687806464 to 7315625068529106944. Current Snapshot: MvccSnapshot[applied={T|T < 7315625068687806464}]
I20260806 18:34:36.549296 20660 mvcc.cc:205] Tried to move back new op lower bound from 7315625068687806464 to 7315625068529106944. Current Snapshot: MvccSnapshot[applied={T|T < 7315625068687806464}]
I20260806 18:34:36.559530 20103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:36.559876 20103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:36.563216 20103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6: Sending DeleteTablet for 3 replicas of tablet 6ab68291a7f341b7a6c5ba7aef6c5803
I20260806 18:34:36.564649 20253 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ab68291a7f341b7a6c5ba7aef6c5803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:36 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:54656
I20260806 18:34:36.564640 20551 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ab68291a7f341b7a6c5ba7aef6c5803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:36 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:57558
I20260806 18:34:36.564733 20405 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ab68291a7f341b7a6c5ba7aef6c5803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:36 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:36088
I20260806 18:34:36.565543 20669 tablet_replica.cc:330] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e: stopping tablet replica
I20260806 18:34:36.565586 20668 tablet_replica.cc:330] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae: stopping tablet replica
I20260806 18:34:36.565955 20669 raft_consensus.cc:2229] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:36.566027 20668 raft_consensus.cc:2229] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:36.566404 20669 raft_consensus.cc:2258] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:36.566565 20103 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:34:36.566720 20668 raft_consensus.cc:2258] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 18:34:36.567139 20103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:36.567464 20667 tablet_replica.cc:330] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a: stopping tablet replica
I20260806 18:34:36.567878 20667 raft_consensus.cc:2229] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:36.568325 20667 raft_consensus.cc:2258] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:36.569308 20668 ts_tablet_manager.cc:1905] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:36.570291 20667 ts_tablet_manager.cc:1905] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:36.571501 20669 ts_tablet_manager.cc:1905] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:36.574385 20405 tablet_service.cc:1467] Processing CreateTablet for tablet 62a3e5b2507c48c98670afed9d8eae63 (DEFAULT_TABLE table=test-table [id=794a59cde1bf4d1bb855dc2ff3114a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:36.574525 20667 ts_tablet_manager.cc:1918] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:36.574729 20405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62a3e5b2507c48c98670afed9d8eae63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:36.575524 20667 log.cc:1196] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/6ab68291a7f341b7a6c5ba7aef6c5803
I20260806 18:34:36.575933 20667 ts_tablet_manager.cc:1939] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 87c334f16375472b8a002c6a2f6bae3a: Deleting consensus metadata
I20260806 18:34:36.576887 20668 ts_tablet_manager.cc:1918] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:36.577003 20253 tablet_service.cc:1467] Processing CreateTablet for tablet 62a3e5b2507c48c98670afed9d8eae63 (DEFAULT_TABLE table=test-table [id=794a59cde1bf4d1bb855dc2ff3114a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:36.577381 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62a3e5b2507c48c98670afed9d8eae63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:36.577615 20551 tablet_service.cc:1467] Processing CreateTablet for tablet 62a3e5b2507c48c98670afed9d8eae63 (DEFAULT_TABLE table=test-table [id=794a59cde1bf4d1bb855dc2ff3114a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:36.577875 20551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62a3e5b2507c48c98670afed9d8eae63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:36.577005 20668 log.cc:1196] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/wal/wals/6ab68291a7f341b7a6c5ba7aef6c5803
I20260806 18:34:36.579032 20668 ts_tablet_manager.cc:1939] T 6ab68291a7f341b7a6c5ba7aef6c5803 P d9939f1e14cd450f99cfda12d85c82ae: Deleting consensus metadata
I20260806 18:34:36.580453 20088 catalog_manager.cc:4841] TS 87c334f16375472b8a002c6a2f6bae3a (127.18.125.131:39679): tablet 6ab68291a7f341b7a6c5ba7aef6c5803 (table test-table [id=c2f412a8b77b4b95895acaffdae6a98c]) successfully deleted
I20260806 18:34:36.580750 20646 tablet_bootstrap.cc:492] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae: Bootstrap starting.
I20260806 18:34:36.581367 20088 catalog_manager.cc:4841] TS d9939f1e14cd450f99cfda12d85c82ae (127.18.125.129:44275): tablet 6ab68291a7f341b7a6c5ba7aef6c5803 (table test-table [id=c2f412a8b77b4b95895acaffdae6a98c]) successfully deleted
I20260806 18:34:36.574453 20669 ts_tablet_manager.cc:1918] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:36.581830 20645 tablet_bootstrap.cc:492] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a: Bootstrap starting.
I20260806 18:34:36.581964 20669 log.cc:1196] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/6ab68291a7f341b7a6c5ba7aef6c5803
I20260806 18:34:36.582078 20646 tablet_bootstrap.cc:654] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:36.582419 20669 ts_tablet_manager.cc:1939] T 6ab68291a7f341b7a6c5ba7aef6c5803 P 6df850a8c0de4df4ba6eac63469c075e: Deleting consensus metadata
I20260806 18:34:36.582903 20645 tablet_bootstrap.cc:654] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:36.583451 20646 tablet_bootstrap.cc:492] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae: No bootstrap required, opened a new log
I20260806 18:34:36.583580 20646 ts_tablet_manager.cc:1397] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:36.584192 20646 raft_consensus.cc:348] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.584321 20646 raft_consensus.cc:374] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:36.584358 20646 raft_consensus.cc:729] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9939f1e14cd450f99cfda12d85c82ae, State: Initialized, Role: FOLLOWER
I20260806 18:34:36.584448 20644 tablet_bootstrap.cc:492] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e: Bootstrap starting.
I20260806 18:34:36.584501 20646 consensus_queue.cc:260] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.584811 20646 ts_tablet_manager.cc:1428] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:34:36.585561 20645 tablet_bootstrap.cc:492] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a: No bootstrap required, opened a new log
I20260806 18:34:36.585685 20645 ts_tablet_manager.cc:1397] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 18:34:36.585825 20644 tablet_bootstrap.cc:654] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:36.585919 20087 catalog_manager.cc:4841] TS 6df850a8c0de4df4ba6eac63469c075e (127.18.125.130:38457): tablet 6ab68291a7f341b7a6c5ba7aef6c5803 (table test-table [id=c2f412a8b77b4b95895acaffdae6a98c]) successfully deleted
I20260806 18:34:36.586230 20645 raft_consensus.cc:348] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.586375 20645 raft_consensus.cc:374] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:36.586436 20645 raft_consensus.cc:729] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87c334f16375472b8a002c6a2f6bae3a, State: Initialized, Role: FOLLOWER
I20260806 18:34:36.586594 20645 consensus_queue.cc:260] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.586916 20645 ts_tablet_manager.cc:1428] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:36.587136 20644 tablet_bootstrap.cc:492] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e: No bootstrap required, opened a new log
I20260806 18:34:36.587234 20644 ts_tablet_manager.cc:1397] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 18:34:36.587743 20644 raft_consensus.cc:348] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.587877 20644 raft_consensus.cc:374] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:36.587932 20644 raft_consensus.cc:729] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df850a8c0de4df4ba6eac63469c075e, State: Initialized, Role: FOLLOWER
I20260806 18:34:36.588061 20644 consensus_queue.cc:260] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.588394 20644 ts_tablet_manager.cc:1428] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260806 18:34:36.588681 20327 tablet.cc:2369] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:34:36.596881 20475 tablet.cc:2369] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:36.685644 20650 raft_consensus.cc:482] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:36.685840 20650 raft_consensus.cc:504] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.686556 20650 leader_election.cc:288] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6df850a8c0de4df4ba6eac63469c075e (127.18.125.130:38457), 87c334f16375472b8a002c6a2f6bae3a (127.18.125.131:39679)
I20260806 18:34:36.687103 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62a3e5b2507c48c98670afed9d8eae63" candidate_uuid: "d9939f1e14cd450f99cfda12d85c82ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c334f16375472b8a002c6a2f6bae3a" is_pre_election: true
I20260806 18:34:36.687117 20425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62a3e5b2507c48c98670afed9d8eae63" candidate_uuid: "d9939f1e14cd450f99cfda12d85c82ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6df850a8c0de4df4ba6eac63469c075e" is_pre_election: true
I20260806 18:34:36.687280 20572 raft_consensus.cc:2454] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9939f1e14cd450f99cfda12d85c82ae in term 0.
I20260806 18:34:36.687280 20425 raft_consensus.cc:2454] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9939f1e14cd450f99cfda12d85c82ae in term 0.
I20260806 18:34:36.687674 20199 leader_election.cc:302] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6df850a8c0de4df4ba6eac63469c075e, d9939f1e14cd450f99cfda12d85c82ae; no voters:
I20260806 18:34:36.687899 20650 raft_consensus.cc:2781] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:36.687981 20650 raft_consensus.cc:482] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:36.688040 20650 raft_consensus.cc:3037] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:36.689100 20650 raft_consensus.cc:504] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.689499 20650 leader_election.cc:288] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [CANDIDATE]: Term 1 election: Requested vote from peers 6df850a8c0de4df4ba6eac63469c075e (127.18.125.130:38457), 87c334f16375472b8a002c6a2f6bae3a (127.18.125.131:39679)
I20260806 18:34:36.689898 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62a3e5b2507c48c98670afed9d8eae63" candidate_uuid: "d9939f1e14cd450f99cfda12d85c82ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c334f16375472b8a002c6a2f6bae3a"
I20260806 18:34:36.690053 20572 raft_consensus.cc:3037] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:36.690112 20425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62a3e5b2507c48c98670afed9d8eae63" candidate_uuid: "d9939f1e14cd450f99cfda12d85c82ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6df850a8c0de4df4ba6eac63469c075e"
I20260806 18:34:36.690256 20425 raft_consensus.cc:3037] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:36.691434 20572 raft_consensus.cc:2454] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9939f1e14cd450f99cfda12d85c82ae in term 1.
I20260806 18:34:36.691720 20425 raft_consensus.cc:2454] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9939f1e14cd450f99cfda12d85c82ae in term 1.
I20260806 18:34:36.691891 20200 leader_election.cc:302] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87c334f16375472b8a002c6a2f6bae3a, d9939f1e14cd450f99cfda12d85c82ae; no voters:
I20260806 18:34:36.692101 20650 raft_consensus.cc:2781] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:36.692260 20650 raft_consensus.cc:686] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 1 LEADER]: Becoming Leader. State: Replica: d9939f1e14cd450f99cfda12d85c82ae, State: Running, Role: LEADER
I20260806 18:34:36.692472 20650 consensus_queue.cc:237] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.694255 20103 catalog_manager.cc:5495] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae reported cstate change: term changed from 0 to 1, leader changed from <none> to d9939f1e14cd450f99cfda12d85c82ae (127.18.125.129). New cstate: current_term: 1 leader_uuid: "d9939f1e14cd450f99cfda12d85c82ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } health_report { overall_health: UNKNOWN } } }
Aug 06 18:34:36 dist-test-slave-8w9v krb5kdc[20052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041275, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:36.731945 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.722159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42104 (local address 127.18.125.190:38387)
0806 18:34:36.722315 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.722320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.722487 (+ 167us) server_negotiation.cc:407] Connection header received
0806 18:34:36.722622 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.722627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.722696 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.722837 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:36.723468 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.724327 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.725129 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.725524 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.725652 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.728210 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:36.728214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:36.728217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:36.728242 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:36.729885 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.730468 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.730471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.730472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.730552 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:36.730873 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:36.730876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:36.730877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:36.731136 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:36.731216 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.731610 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.731804 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":53}
I20260806 18:34:36.731950 20679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.722012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:38387 (local address 127.0.0.1:42104)
0806 18:34:36.722390 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.722409 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.722521 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.722864 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.722866 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.723248 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:36.723347 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.723352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.724444 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.724446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.724998 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.725005 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.725647 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.725650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.725710 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.726310 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:36.726323 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:36.728035 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:36.729999 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:36.730010 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:36.730018 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:36.730338 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:36.730686 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:36.730689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:36.730691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:36.730779 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:36.731249 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:36.731260 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:36.731425 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.731556 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.731723 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":152,"threads_started":1}
I20260806 18:34:36.736585 20103 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:42104:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:34:36.737417 20103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:36.743522 20405 tablet_service.cc:1467] Processing CreateTablet for tablet 0a124997a018417b925ef20855b15918 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e637c7515cee44d893431299114cd833]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:36.743523 20253 tablet_service.cc:1467] Processing CreateTablet for tablet 0a124997a018417b925ef20855b15918 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e637c7515cee44d893431299114cd833]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:36.743651 20551 tablet_service.cc:1467] Processing CreateTablet for tablet 0a124997a018417b925ef20855b15918 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e637c7515cee44d893431299114cd833]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:36.743954 20405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a124997a018417b925ef20855b15918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:36.743963 20253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a124997a018417b925ef20855b15918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:36.744021 20551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a124997a018417b925ef20855b15918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:36.746616 20644 tablet_bootstrap.cc:492] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: Bootstrap starting.
I20260806 18:34:36.747149 20646 tablet_bootstrap.cc:492] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae: Bootstrap starting.
I20260806 18:34:36.747728 20644 tablet_bootstrap.cc:654] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:36.748199 20646 tablet_bootstrap.cc:654] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:36.748852 20645 tablet_bootstrap.cc:492] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: Bootstrap starting.
I20260806 18:34:36.749440 20646 tablet_bootstrap.cc:492] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae: No bootstrap required, opened a new log
I20260806 18:34:36.749439 20644 tablet_bootstrap.cc:492] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: No bootstrap required, opened a new log
I20260806 18:34:36.749539 20644 ts_tablet_manager.cc:1397] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:36.749539 20646 ts_tablet_manager.cc:1397] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:36.749796 20645 tablet_bootstrap.cc:654] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:36.750022 20644 raft_consensus.cc:348] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.750022 20646 raft_consensus.cc:348] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.750156 20644 raft_consensus.cc:374] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:36.750156 20646 raft_consensus.cc:374] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:36.750191 20644 raft_consensus.cc:729] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df850a8c0de4df4ba6eac63469c075e, State: Initialized, Role: FOLLOWER
I20260806 18:34:36.750191 20646 raft_consensus.cc:729] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9939f1e14cd450f99cfda12d85c82ae, State: Initialized, Role: FOLLOWER
I20260806 18:34:36.750327 20644 consensus_queue.cc:260] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.750384 20646 consensus_queue.cc:260] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.750614 20644 ts_tablet_manager.cc:1428] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:34:36.750660 20646 ts_tablet_manager.cc:1428] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:36.750658 20652 tablet_replica.cc:439] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } }
I20260806 18:34:36.750726 20650 tablet_replica.cc:439] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } }
I20260806 18:34:36.750819 20652 tablet_replica.cc:442] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:36.750833 20650 tablet_replica.cc:442] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:36.752540 20645 tablet_bootstrap.cc:492] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: No bootstrap required, opened a new log
I20260806 18:34:36.752691 20645 ts_tablet_manager.cc:1397] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 18:34:36.753429 20645 raft_consensus.cc:348] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.753597 20645 raft_consensus.cc:374] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:36.753640 20645 raft_consensus.cc:729] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87c334f16375472b8a002c6a2f6bae3a, State: Initialized, Role: FOLLOWER
I20260806 18:34:36.753760 20645 consensus_queue.cc:260] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.754304 20645 ts_tablet_manager.cc:1428] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 18:34:36.754304 20651 tablet_replica.cc:439] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } }
I20260806 18:34:36.754674 20651 tablet_replica.cc:442] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 18:34:36.835609 20621 tablet.cc:2369] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:36.909106 20652 raft_consensus.cc:482] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:36.909278 20652 raft_consensus.cc:504] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.910554 20652 leader_election.cc:288] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9939f1e14cd450f99cfda12d85c82ae (127.18.125.129:44275), 87c334f16375472b8a002c6a2f6bae3a (127.18.125.131:39679)
I20260806 18:34:36.915246 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.911382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:52347 (local address 127.18.125.131:39679)
0806 18:34:36.911559 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.911564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.911680 (+ 116us) server_negotiation.cc:407] Connection header received
0806 18:34:36.911839 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.911841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.911881 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.911971 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:36.912311 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.912939 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.914085 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.914575 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.914720 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.914783 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.915005 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.915113 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:34:36.915247 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.911255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:39679 (local address 127.18.125.130:52347)
0806 18:34:36.911586 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.911598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.911722 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.911984 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.911987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.912005 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:36.912186 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.912193 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.913054 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.913058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.913950 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.913960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.914715 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.914719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.914826 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.914851 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.914961 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.915057 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":3,"thread_start_us":58,"threads_started":1}
I20260806 18:34:36.915355 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.910966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:42291 (local address 127.18.125.129:44275)
0806 18:34:36.911141 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.911149 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.911261 (+ 112us) server_negotiation.cc:407] Connection header received
0806 18:34:36.911696 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.911727 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.911795 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.911898 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:36.912326 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.913046 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.914171 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.914673 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.914810 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.914868 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.915136 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.915220 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 18:34:36.915700 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.910823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44275 (local address 127.18.125.130:42291)
0806 18:34:36.911136 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.911156 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.911349 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.911915 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.911918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.911939 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:36.912181 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.912191 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.913191 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.913197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.914034 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.914043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.914810 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.914814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.914975 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.915000 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.915504 (+ 504us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.915589 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":22,"thread_start_us":98,"threads_started":1}
I20260806 18:34:36.916079 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a124997a018417b925ef20855b15918" candidate_uuid: "6df850a8c0de4df4ba6eac63469c075e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c334f16375472b8a002c6a2f6bae3a" is_pre_election: true
I20260806 18:34:36.916316 20572 raft_consensus.cc:2454] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6df850a8c0de4df4ba6eac63469c075e in term 0.
I20260806 18:34:36.916335 20276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a124997a018417b925ef20855b15918" candidate_uuid: "6df850a8c0de4df4ba6eac63469c075e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9939f1e14cd450f99cfda12d85c82ae" is_pre_election: true
I20260806 18:34:36.916604 20276 raft_consensus.cc:2454] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6df850a8c0de4df4ba6eac63469c075e in term 0.
I20260806 18:34:36.916742 20354 leader_election.cc:302] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6df850a8c0de4df4ba6eac63469c075e, 87c334f16375472b8a002c6a2f6bae3a; no voters:
I20260806 18:34:36.917085 20652 raft_consensus.cc:2781] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:36.917158 20652 raft_consensus.cc:482] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:36.917197 20652 raft_consensus.cc:3037] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:36.918277 20652 raft_consensus.cc:504] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.918653 20652 leader_election.cc:288] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [CANDIDATE]: Term 1 election: Requested vote from peers d9939f1e14cd450f99cfda12d85c82ae (127.18.125.129:44275), 87c334f16375472b8a002c6a2f6bae3a (127.18.125.131:39679)
I20260806 18:34:36.919027 20276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a124997a018417b925ef20855b15918" candidate_uuid: "6df850a8c0de4df4ba6eac63469c075e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9939f1e14cd450f99cfda12d85c82ae"
I20260806 18:34:36.919085 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a124997a018417b925ef20855b15918" candidate_uuid: "6df850a8c0de4df4ba6eac63469c075e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c334f16375472b8a002c6a2f6bae3a"
I20260806 18:34:36.919173 20276 raft_consensus.cc:3037] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:36.919196 20572 raft_consensus.cc:3037] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:36.920475 20572 raft_consensus.cc:2454] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6df850a8c0de4df4ba6eac63469c075e in term 1.
I20260806 18:34:36.920573 20276 raft_consensus.cc:2454] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6df850a8c0de4df4ba6eac63469c075e in term 1.
I20260806 18:34:36.921099 20354 leader_election.cc:302] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6df850a8c0de4df4ba6eac63469c075e, d9939f1e14cd450f99cfda12d85c82ae; no voters:
I20260806 18:34:36.921321 20652 raft_consensus.cc:2781] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:36.921557 20652 raft_consensus.cc:686] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [term 1 LEADER]: Becoming Leader. State: Replica: 6df850a8c0de4df4ba6eac63469c075e, State: Running, Role: LEADER
I20260806 18:34:36.921885 20652 consensus_queue.cc:237] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } }
I20260806 18:34:36.923619 20687 tablet_replica.cc:439] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: TxnStatusTablet state changed. Reason: New leader 6df850a8c0de4df4ba6eac63469c075e. Latest consensus state: current_term: 1 leader_uuid: "6df850a8c0de4df4ba6eac63469c075e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } }
I20260806 18:34:36.923740 20687 tablet_replica.cc:442] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:36.924057 20689 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:34:36.924259 20689 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:34:36.924922 20103 catalog_manager.cc:5495] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6df850a8c0de4df4ba6eac63469c075e (127.18.125.130). New cstate: current_term: 1 leader_uuid: "6df850a8c0de4df4ba6eac63469c075e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:36.954298 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.949913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36104 (local address 127.18.125.130:38457)
0806 18:34:36.950059 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.950063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.950159 (+ 96us) server_negotiation.cc:407] Connection header received
0806 18:34:36.950400 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.950403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.950463 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.950547 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:36.950851 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.951348 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.952618 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.952840 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.952922 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.953293 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:36.953886 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:36.953979 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.954103 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.954157 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:34:36.954334 20679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.949788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:38457 (local address 127.0.0.1:36104)
0806 18:34:36.950077 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.950093 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.950223 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.950591 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.950594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.950601 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:36.950723 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.950730 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.951459 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.951462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.952493 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.952505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.952962 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.952966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.953065 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.953132 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:36.953984 (+ 852us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:36.954002 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.954082 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.954181 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260806 18:34:36.957581 20572 raft_consensus.cc:1261] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a [term 1 FOLLOWER]: Refusing update from remote peer 6df850a8c0de4df4ba6eac63469c075e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:36.957681 20276 raft_consensus.cc:1261] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae [term 1 FOLLOWER]: Refusing update from remote peer 6df850a8c0de4df4ba6eac63469c075e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:36.958180 20652 consensus_queue.cc:1035] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [LEADER]: Connected to new peer: Peer: permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:36.958405 20687 consensus_queue.cc:1035] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:36.960220 20650 tablet_replica.cc:439] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae: TxnStatusTablet state changed. Reason: New leader 6df850a8c0de4df4ba6eac63469c075e. Latest consensus state: current_term: 1 leader_uuid: "6df850a8c0de4df4ba6eac63469c075e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } }
I20260806 18:34:36.960224 20651 tablet_replica.cc:439] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: TxnStatusTablet state changed. Reason: New leader 6df850a8c0de4df4ba6eac63469c075e. Latest consensus state: current_term: 1 leader_uuid: "6df850a8c0de4df4ba6eac63469c075e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } }
I20260806 18:34:36.960350 20650 tablet_replica.cc:442] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:36.960355 20651 tablet_replica.cc:442] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:36.962715 20687 tablet_replica.cc:439] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6df850a8c0de4df4ba6eac63469c075e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } }
I20260806 18:34:36.962842 20687 tablet_replica.cc:442] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:36.964619 20651 tablet_replica.cc:439] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6df850a8c0de4df4ba6eac63469c075e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } }
I20260806 18:34:36.964610 20652 tablet_replica.cc:439] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6df850a8c0de4df4ba6eac63469c075e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } }
I20260806 18:34:36.964731 20651 tablet_replica.cc:442] T 0a124997a018417b925ef20855b15918 P 87c334f16375472b8a002c6a2f6bae3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:36.964731 20652 tablet_replica.cc:442] T 0a124997a018417b925ef20855b15918 P 6df850a8c0de4df4ba6eac63469c075e: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:36.966535 20658 mvcc.cc:205] Tried to move back new op lower bound from 7315625070411788288 to 7315625070275448832. Current Snapshot: MvccSnapshot[applied={T|T < 7315625070411788288}]
I20260806 18:34:36.966738 20650 tablet_replica.cc:439] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6df850a8c0de4df4ba6eac63469c075e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9939f1e14cd450f99cfda12d85c82ae" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44275 } } peers { permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 } } peers { permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 } } }
I20260806 18:34:36.966840 20650 tablet_replica.cc:442] T 0a124997a018417b925ef20855b15918 P d9939f1e14cd450f99cfda12d85c82ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:36.981108 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.976980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:38457 (local address 127.18.125.129:36989)
0806 18:34:36.977322 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:36.977337 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:36.977445 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:36.977750 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:36.977752 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:36.977759 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:36.977864 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.977869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.978943 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.978946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.979595 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:36.979603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.980299 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.980302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:36.980381 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.980446 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:36.980812 (+ 366us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:36.980818 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:34:36.980874 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 18:34:36.980962 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":145,"threads_started":1}
I20260806 18:34:36.981153 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:36.977116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:36989 (local address 127.18.125.130:38457)
0806 18:34:36.977272 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:36.977278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:36.977418 (+ 140us) server_negotiation.cc:407] Connection header received
0806 18:34:36.977554 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:36.977560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:36.977627 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:36.977738 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:36.977983 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.978814 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.979729 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:36.980094 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:36.980260 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:36.980600 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:36.980709 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:36.980798 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:36.980918 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 18:34:36.980986 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 18:34:36.989450 20572 raft_consensus.cc:1261] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a [term 1 FOLLOWER]: Refusing update from remote peer d9939f1e14cd450f99cfda12d85c82ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:36.989533 20425 raft_consensus.cc:1261] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e [term 1 FOLLOWER]: Refusing update from remote peer d9939f1e14cd450f99cfda12d85c82ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:36.989915 20650 consensus_queue.cc:1035] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "6df850a8c0de4df4ba6eac63469c075e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:36.990091 20655 consensus_queue.cc:1035] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "87c334f16375472b8a002c6a2f6bae3a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:37.016983 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.012539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:44905 (local address 127.18.125.129:44275)
0806 18:34:37.012716 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.012722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.013055 (+ 333us) server_negotiation.cc:407] Connection header received
0806 18:34:37.013300 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.013304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.013366 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.013419 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:37.013867 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.014659 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.015677 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.015935 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.016074 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.016434 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:37.016546 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:37.016632 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.016770 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.016853 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:34:37.017509 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.012397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44275 (local address 127.18.125.130:44905)
0806 18:34:37.012959 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:37.012977 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:37.013145 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:37.013576 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:37.013579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:37.013587 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:37.013744 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.013753 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.014833 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.014835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.015532 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:37.015543 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.016077 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.016081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:37.016219 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.016274 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:37.016641 (+ 367us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:37.016651 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 18:34:37.017234 (+ 583us) client_negotiation.cc:240] Negotiation successful
0806 18:34:37.017330 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":131,"threads_started":1}
I20260806 18:34:37.047073 20102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:37.047240 20102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:37.049835 20102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a0f45b54f2b4ca5bf8dc141ce5179f6: Sending DeleteTablet for 3 replicas of tablet 62a3e5b2507c48c98670afed9d8eae63
I20260806 18:34:37.050541 20253 tablet_service.cc:1514] Processing DeleteTablet for tablet 62a3e5b2507c48c98670afed9d8eae63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:37 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:54656
I20260806 18:34:37.050689 20405 tablet_service.cc:1514] Processing DeleteTablet for tablet 62a3e5b2507c48c98670afed9d8eae63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:37 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:36088
I20260806 18:34:37.050778 20668 tablet_replica.cc:330] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae: stopping tablet replica
I20260806 18:34:37.050861 20669 tablet_replica.cc:330] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e: stopping tablet replica
I20260806 18:34:37.050906 20668 raft_consensus.cc:2229] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:37.050971 20669 raft_consensus.cc:2229] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:37.051095 20669 raft_consensus.cc:2258] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:37.051120 20668 raft_consensus.cc:2258] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:37.051503 20551 tablet_service.cc:1514] Processing DeleteTablet for tablet 62a3e5b2507c48c98670afed9d8eae63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:37 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:57558
I20260806 18:34:37.051715 20667 tablet_replica.cc:330] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a: stopping tablet replica
I20260806 18:34:37.051863 20667 raft_consensus.cc:2229] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:37.052013 20667 raft_consensus.cc:2258] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:37.052162 20669 ts_tablet_manager.cc:1905] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:37.052837 20668 ts_tablet_manager.cc:1905] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:37.053170 20667 ts_tablet_manager.cc:1905] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:37.055506 20669 ts_tablet_manager.cc:1918] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:37.055591 20669 log.cc:1196] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/62a3e5b2507c48c98670afed9d8eae63
I20260806 18:34:37.055835 20668 ts_tablet_manager.cc:1918] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:37.055903 20669 ts_tablet_manager.cc:1939] T 62a3e5b2507c48c98670afed9d8eae63 P 6df850a8c0de4df4ba6eac63469c075e: Deleting consensus metadata
I20260806 18:34:37.055917 20668 log.cc:1196] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-0/wal/wals/62a3e5b2507c48c98670afed9d8eae63
I20260806 18:34:37.056066 20667 ts_tablet_manager.cc:1918] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:37.056164 20667 log.cc:1196] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/62a3e5b2507c48c98670afed9d8eae63
I20260806 18:34:37.056257 20668 ts_tablet_manager.cc:1939] T 62a3e5b2507c48c98670afed9d8eae63 P d9939f1e14cd450f99cfda12d85c82ae: Deleting consensus metadata
I20260806 18:34:37.056435 20667 ts_tablet_manager.cc:1939] T 62a3e5b2507c48c98670afed9d8eae63 P 87c334f16375472b8a002c6a2f6bae3a: Deleting consensus metadata
I20260806 18:34:37.056869 20087 catalog_manager.cc:4841] TS 6df850a8c0de4df4ba6eac63469c075e (127.18.125.130:38457): tablet 62a3e5b2507c48c98670afed9d8eae63 (table test-table [id=794a59cde1bf4d1bb855dc2ff3114a76]) successfully deleted
I20260806 18:34:37.057166 20088 catalog_manager.cc:4841] TS 87c334f16375472b8a002c6a2f6bae3a (127.18.125.131:39679): tablet 62a3e5b2507c48c98670afed9d8eae63 (table test-table [id=794a59cde1bf4d1bb855dc2ff3114a76]) successfully deleted
I20260806 18:34:37.057343 20088 catalog_manager.cc:4841] TS d9939f1e14cd450f99cfda12d85c82ae (127.18.125.129:44275): tablet 62a3e5b2507c48c98670afed9d8eae63 (table test-table [id=794a59cde1bf4d1bb855dc2ff3114a76]) successfully deleted
Aug 06 18:34:37 dist-test-slave-8w9v krb5kdc[20052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:34:37.069460 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.058101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54674 (local address 127.18.125.129:44275)
0806 18:34:37.058231 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.058238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.058387 (+ 149us) server_negotiation.cc:407] Connection header received
0806 18:34:37.058559 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.058561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.058609 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.058678 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.059658 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.060263 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.061478 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.061715 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.061802 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.065277 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.065293 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.065310 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.065342 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.067341 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.067858 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.067872 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.067874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.067974 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.068317 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.068321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.068322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.068638 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.068891 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.069066 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.069285 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 18:34:37.069973 20310 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54674
I20260806 18:34:37.080431 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:37.071994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.18.125.190:38387)
0806 18:34:37.072168 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:37.072174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:37.072404 (+ 230us) server_negotiation.cc:407] Connection header received
0806 18:34:37.072536 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:37.072538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:37.072587 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:37.072653 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:37.073513 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.074049 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.075241 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:37.075454 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:37.075529 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:37.077113 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:37.077118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:37.077120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:37.077154 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:37.078644 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.079099 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.079104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.079106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.079192 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:37.079483 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:37.079488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:37.079490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:37.079692 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:37.079837 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:37.080081 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 18:34:37.080251 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 18:34:37.080938 20102 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42112
I20260806 18:34:37.082211 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 20182
I20260806 18:34:37.097465 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 20335
I20260806 18:34:37.106079 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 20483
I20260806 18:34:37.114317 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 20069
2026-08-06T18:34:37Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3064 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:37 dist-test-slave-8w9v krb5kdc[20710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:37 dist-test-slave-8w9v krb5kdc[20710](info): set up 2 sockets
Aug 06 18:34:37 dist-test-slave-8w9v krb5kdc[20710](info): commencing operation
krb5kdc: starting...
W20260806 18:34:38.213948 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:34:38 dist-test-slave-8w9v krb5kdc[20710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:34:38.349082 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:42317
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:42317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:38.467702 20723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:38.467993 20723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:38.468037 20723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:38.472177 20723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:38.472244 20723 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:34:38.472261 20723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:38.472280 20723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:38.472296 20723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:38.472311 20723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:38.478735 20723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:42317
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:42317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:38.479995 20723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:38.481489 20723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:38.491648 20733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.491688 20731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.491648 20730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:38.491972 20723 server_base.cc:1034] running on GCE node
I20260806 18:34:38.492745 20723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:38.493994 20723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:38.495190 20723 hybrid_clock.cc:648] HybridClock initialized: now 1786041278495170 us; error 34 us; skew 500 ppm
Aug 06 18:34:38 dist-test-slave-8w9v krb5kdc[20710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:38.498514 20723 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:34:38.499586 20723 webserver.cc:460] Webserver started at http://127.18.125.190:33729/ using document root <none> and password file <none>
I20260806 18:34:38.500283 20723 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:38.500360 20723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:38.500613 20723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:38.502936 20723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "bb5bb9c1be9f4c1ba85db1b18365a822"
format_stamp: "Formatted at 2026-08-06 18:34:38 on dist-test-slave-8w9v"
server_key: "9968c44e663f9d98f995fba9b9095fbe"
server_key_iv: "8c1bcaa0b77f880be21173619dc0608f"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.503636 20723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "bb5bb9c1be9f4c1ba85db1b18365a822"
format_stamp: "Formatted at 2026-08-06 18:34:38 on dist-test-slave-8w9v"
server_key: "9968c44e663f9d98f995fba9b9095fbe"
server_key_iv: "8c1bcaa0b77f880be21173619dc0608f"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.508464 20723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 18:34:38.511260 20741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.512744 20723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:38.512914 20723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "bb5bb9c1be9f4c1ba85db1b18365a822"
format_stamp: "Formatted at 2026-08-06 18:34:38 on dist-test-slave-8w9v"
server_key: "9968c44e663f9d98f995fba9b9095fbe"
server_key_iv: "8c1bcaa0b77f880be21173619dc0608f"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.513053 20723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:38.554407 20723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:38.558229 20723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:38.558466 20723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:38.567929 20723 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:42317
I20260806 18:34:38.567945 20806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:42317 every 8 connection(s)
I20260806 18:34:38.569090 20723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:38.572220 20807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:38.577708 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 20723
I20260806 18:34:38.577857 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:38.578120 18934 external_mini_cluster.cc:1428] Setting key b342ee644c15b7b2d3bfd18393237594
I20260806 18:34:38.579310 20807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822: Bootstrap starting.
I20260806 18:34:38.582204 20807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:38.583148 20807 log.cc:824] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:38.585497 20807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822: No bootstrap required, opened a new log
Aug 06 18:34:38 dist-test-slave-8w9v krb5kdc[20710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:38.590059 20807 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5bb9c1be9f4c1ba85db1b18365a822" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42317 } }
I20260806 18:34:38.590392 20807 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:38.590463 20807 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb5bb9c1be9f4c1ba85db1b18365a822, State: Initialized, Role: FOLLOWER
I20260806 18:34:38.591008 20807 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5bb9c1be9f4c1ba85db1b18365a822" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42317 } }
I20260806 18:34:38.591163 20807 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:38.591261 20807 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:38.591365 20807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:38.592437 20807 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5bb9c1be9f4c1ba85db1b18365a822" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42317 } }
I20260806 18:34:38.592804 20807 leader_election.cc:302] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb5bb9c1be9f4c1ba85db1b18365a822; no voters:
I20260806 18:34:38.593174 20807 leader_election.cc:288] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:38.593385 20812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:38.593628 20812 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [term 1 LEADER]: Becoming Leader. State: Replica: bb5bb9c1be9f4c1ba85db1b18365a822, State: Running, Role: LEADER
I20260806 18:34:38.593979 20812 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5bb9c1be9f4c1ba85db1b18365a822" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42317 } }
I20260806 18:34:38.594332 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.579706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.18.125.190:42317)
0806 18:34:38.580196 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.580216 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.580259 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:34:38.581193 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.581218 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.581674 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.582043 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.582849 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.584263 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.585255 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.585961 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.586129 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.588787 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.588795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.588815 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.588852 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.590895 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.591458 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.591465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.591471 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.591598 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.591906 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.591909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.591911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.592415 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.592628 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.592923 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.593259 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":139,"threads_started":1}
I20260806 18:34:38.594638 20807 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:38.595302 20812 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb5bb9c1be9f4c1ba85db1b18365a822. Latest consensus state: current_term: 1 leader_uuid: "bb5bb9c1be9f4c1ba85db1b18365a822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5bb9c1be9f4c1ba85db1b18365a822" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42317 } } }
I20260806 18:34:38.595424 20812 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:38.595845 20814 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb5bb9c1be9f4c1ba85db1b18365a822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5bb9c1be9f4c1ba85db1b18365a822" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42317 } } }
I20260806 18:34:38.595945 20814 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:38.596764 20822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:38.599714 20822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:38.605330 20822 catalog_manager.cc:1347] Generated new cluster ID: 9e3d2f8843f347cd8c89704f891574bf
I20260806 18:34:38.605423 20822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:38.633534 20822 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:38.634931 20822 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:38.646503 20822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822: Generated new TSK 0
I20260806 18:34:38.647508 20822 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:34:38.683720 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42317
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:38.820312 20835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:38.820634 20835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:38.820689 20835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:38.825085 20835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:38.825176 20835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:38.825222 20835 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:34:38.825237 20835 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:38.825340 20835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:34:38.831163 20835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.125.190:42317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:38.832659 20835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:38.834111 20835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:34:38.846280 20835 server_base.cc:1034] running on GCE node
W20260806 18:34:38.846325 20840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.846458 20843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:38.846325 20841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:38.847055 20835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:38.847767 20835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:38.848938 20835 hybrid_clock.cc:648] HybridClock initialized: now 1786041278848923 us; error 36 us; skew 500 ppm
Aug 06 18:34:38 dist-test-slave-8w9v krb5kdc[20710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:38.852002 20835 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:34:38.853096 20835 webserver.cc:460] Webserver started at http://127.18.125.129:33493/ using document root <none> and password file <none>
I20260806 18:34:38.853763 20835 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:38.853865 20835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:38.854117 20835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:38.856408 20835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "770f7edf17934c03a6f0fcf28c7943ec"
format_stamp: "Formatted at 2026-08-06 18:34:38 on dist-test-slave-8w9v"
server_key: "413aa1f81a89eaa63cb8adf6bb7c9c02"
server_key_iv: "247c2de914b7d9e124ab57910390f3ed"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.857000 20835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "770f7edf17934c03a6f0fcf28c7943ec"
format_stamp: "Formatted at 2026-08-06 18:34:38 on dist-test-slave-8w9v"
server_key: "413aa1f81a89eaa63cb8adf6bb7c9c02"
server_key_iv: "247c2de914b7d9e124ab57910390f3ed"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.861040 20835 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:34:38.863626 20850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.865180 20835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:34:38.865383 20835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "770f7edf17934c03a6f0fcf28c7943ec"
format_stamp: "Formatted at 2026-08-06 18:34:38 on dist-test-slave-8w9v"
server_key: "413aa1f81a89eaa63cb8adf6bb7c9c02"
server_key_iv: "247c2de914b7d9e124ab57910390f3ed"
server_key_version: "encryptionkey@0"
I20260806 18:34:38.865526 20835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:38.887840 20835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:38.891199 20835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:38.891435 20835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:38.893110 20835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:38.893193 20835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:38.893267 20835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:38.893322 20835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:38 dist-test-slave-8w9v krb5kdc[20710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:38.905092 20835 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:32807
I20260806 18:34:38.905128 20970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:32807 every 8 connection(s)
I20260806 18:34:38.906392 20835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:38.908579 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.893911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:49525 (local address 127.18.125.190:42317)
0806 18:34:38.894072 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.894077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.895087 (+ 1010us) server_negotiation.cc:407] Connection header received
0806 18:34:38.896443 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.896447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.896516 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.896617 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.898011 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.898739 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.900003 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.900341 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.900466 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.904349 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.904354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.904356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.904394 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.906164 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.906826 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.906830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.906832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.906954 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.907360 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.907363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.907364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.907575 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.907666 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.908318 (+ 652us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.908447 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:34:38.909438 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.894371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42317 (local address 127.18.125.129:49525)
0806 18:34:38.894897 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.894959 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.896201 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.896925 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.896932 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.897345 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.897846 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.897860 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.898873 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.898877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.899864 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.899875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.900539 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.900543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.900729 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.901563 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.901585 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.904091 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.906325 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.906330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.906343 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.906709 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.907075 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.907078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.907080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.907259 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.907713 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.907719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.907990 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.908257 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.908551 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":116,"threads_started":1}
I20260806 18:34:38.910933 20971 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42317
I20260806 18:34:38.911108 20971 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:38.911677 20971 heartbeater.cc:511] Master 127.18.125.190:42317 requested a full tablet report, sending...
I20260806 18:34:38.912631 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 20835
I20260806 18:34:38.912808 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:38.913152 18934 external_mini_cluster.cc:1428] Setting key 6b108bd230a3c08c169287dc9156b628
I20260806 18:34:38.913601 20757 ts_manager.cc:194] Registered new tserver with Master: 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807)
I20260806 18:34:38.914954 20757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:49525
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:34:38.924721 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.913752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:45855 (local address 127.18.125.190:42317)
0806 18:34:38.914297 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:38.914302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:38.914315 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:38.914351 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:38.914354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:38.914396 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:38.914490 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:38.916914 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.917635 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.919107 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.919439 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.919548 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.921137 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:38.921140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:38.921142 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:38.921169 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:38.922814 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.923385 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.923388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.923389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.923478 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:38.923790 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:38.923793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:38.923794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:38.924060 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:38.924217 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:38.924467 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 18:34:38.924584 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457}
I20260806 18:34:38.924728 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:38.913369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42317 (local address 127.18.125.129:45855)
0806 18:34:38.913756 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:38.913775 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:38.913875 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:38.916253 (+ 2378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:38.916258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:38.916537 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:38.916735 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.916747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.917785 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.917793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.918955 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:38.918972 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:38.919592 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:38.919598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:38.919713 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:38.920377 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:38.920399 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:38.920958 (+ 559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:38.922957 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.922961 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.922963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.923269 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.923578 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:38.923582 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:38.923584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:38.923688 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:38.924207 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:38.924211 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:38.924315 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 18:34:38.924409 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:34:38.924550 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:34:38.947687 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42317
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:39.071935 20980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:39.072329 20980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:39.072399 20980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:39.078094 20980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:39.078183 20980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:39.078234 20980 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:34:39.078249 20980 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:39.078284 20980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:34:39.084705 20980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.125.190:42317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:39.085851 20980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:39.086913 20980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:39.098241 20985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:39.098245 20988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:39.098245 20986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:39.098752 20980 server_base.cc:1034] running on GCE node
I20260806 18:34:39.099349 20980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:39.100196 20980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:39.101420 20980 hybrid_clock.cc:648] HybridClock initialized: now 1786041279101410 us; error 48 us; skew 500 ppm
Aug 06 18:34:39 dist-test-slave-8w9v krb5kdc[20710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041279, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:39.104578 20980 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:34:39.105584 20980 webserver.cc:460] Webserver started at http://127.18.125.130:45359/ using document root <none> and password file <none>
I20260806 18:34:39.106634 20980 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:39.106791 20980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:39.107170 20980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:39.109746 20980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "c3e7949fb6e94c088283c87b4e716c25"
format_stamp: "Formatted at 2026-08-06 18:34:39 on dist-test-slave-8w9v"
server_key: "dfc8f708029a65d658e37467d5ab3d63"
server_key_iv: "cd2400a0ef440aea3cc871ba03180b42"
server_key_version: "encryptionkey@0"
I20260806 18:34:39.110416 20980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "c3e7949fb6e94c088283c87b4e716c25"
format_stamp: "Formatted at 2026-08-06 18:34:39 on dist-test-slave-8w9v"
server_key: "dfc8f708029a65d658e37467d5ab3d63"
server_key_iv: "cd2400a0ef440aea3cc871ba03180b42"
server_key_version: "encryptionkey@0"
I20260806 18:34:39.114822 20980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 18:34:39.117921 20996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:39.119294 20980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:34:39.119464 20980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "c3e7949fb6e94c088283c87b4e716c25"
format_stamp: "Formatted at 2026-08-06 18:34:39 on dist-test-slave-8w9v"
server_key: "dfc8f708029a65d658e37467d5ab3d63"
server_key_iv: "cd2400a0ef440aea3cc871ba03180b42"
server_key_version: "encryptionkey@0"
I20260806 18:34:39.119596 20980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:39.157464 20980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:39.160758 20980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:39.160974 20980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:39.162483 20980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:39.162557 20980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:39.162628 20980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:39.162673 20980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:39 dist-test-slave-8w9v krb5kdc[20710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041279, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:39.174311 20980 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:38845
I20260806 18:34:39.174450 21119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:38845 every 8 connection(s)
I20260806 18:34:39.175345 20980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:39.175808 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 20980
I20260806 18:34:39.175942 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:39.176317 18934 external_mini_cluster.cc:1428] Setting key f5e2dd2228b04ffc72c95e4dff811749
I20260806 18:34:39.178066 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.163639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:38265 (local address 127.18.125.190:42317)
0806 18:34:39.163795 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.163800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.164746 (+ 946us) server_negotiation.cc:407] Connection header received
0806 18:34:39.166171 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.166177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.166247 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.166347 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:39.168067 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.168902 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.170262 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.170508 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.170646 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.173741 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:39.173745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:39.173747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:39.173773 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:39.175619 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.176377 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.176381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.176383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.176480 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.176893 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.176896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.176898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.177164 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:39.177276 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.177838 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.177946 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:34:39.179212 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.164005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42317 (local address 127.18.125.130:38265)
0806 18:34:39.164564 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.164624 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.165874 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.166569 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.166580 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.167102 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:39.167873 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.167909 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.169039 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.169042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.170132 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.170143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.170716 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.170724 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.170969 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.171601 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:39.171625 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:39.173558 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:39.175785 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:39.175793 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:39.175809 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:39.176232 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:39.176584 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:39.176586 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:39.176588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:39.176761 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:39.177265 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:39.177270 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:39.177528 (+ 258us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.177729 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.178078 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":176,"threads_started":1}
I20260806 18:34:39.180763 21120 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42317
I20260806 18:34:39.180924 21120 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:39.181563 21120 heartbeater.cc:511] Master 127.18.125.190:42317 requested a full tablet report, sending...
I20260806 18:34:39.182849 20758 ts_manager.cc:194] Registered new tserver with Master: c3e7949fb6e94c088283c87b4e716c25 (127.18.125.130:38845)
I20260806 18:34:39.184283 20758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:38265
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
I20260806 18:34:39.192914 21127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.183723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42317 (local address 127.18.125.130:53617)
0806 18:34:39.184263 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.184281 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.184388 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.184865 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.184870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.185142 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:39.185421 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.185431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.186815 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.186818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.187738 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.187749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.188252 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.188255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.188366 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.188864 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:39.188876 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:39.189219 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:39.191061 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:39.191067 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:39.191072 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:39.191350 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:39.191621 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:39.191626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:39.191631 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:39.191742 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:39.192237 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:39.192243 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:39.192402 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.192548 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.192747 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":49,"thread_start_us":116,"threads_started":1}
I20260806 18:34:39.192989 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.184273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:53617 (local address 127.18.125.190:42317)
0806 18:34:39.184441 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.184448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.184467 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:34:39.184516 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.184522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.184578 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.184675 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:39.185669 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.186622 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.187859 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.188095 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.188246 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.189350 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:39.189353 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:39.189355 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:39.189380 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:39.190941 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.191449 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.191452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.191453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.191531 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.191833 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.191835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.191836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.192073 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:39.192226 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.192637 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.192786 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:34:39.212262 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42317
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:38191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:39.338696 21129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:39.339006 21129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:39.339085 21129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:39.343283 21129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:39.343366 21129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:39.343434 21129 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:34:39.343470 21129 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:34:39.343542 21129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:34:39.349536 21129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:38191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.125.190:42317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:39.350771 21129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:39.351881 21129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:39.362210 21134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:39.362305 21135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:39.362268 21137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:39.362438 21129 server_base.cc:1034] running on GCE node
I20260806 18:34:39.363026 21129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:39.363858 21129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:39.365087 21129 hybrid_clock.cc:648] HybridClock initialized: now 1786041279365056 us; error 42 us; skew 500 ppm
Aug 06 18:34:39 dist-test-slave-8w9v krb5kdc[20710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041279, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:39.368856 21129 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:34:39.370007 21129 webserver.cc:460] Webserver started at http://127.18.125.131:36115/ using document root <none> and password file <none>
I20260806 18:34:39.370657 21129 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:39.370730 21129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:39.370961 21129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:39.373291 21129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "2e4bbef384f34b9ea41ce256fe9c08cc"
format_stamp: "Formatted at 2026-08-06 18:34:39 on dist-test-slave-8w9v"
server_key: "afea61f1183f73fe0d0dec4b18eab3a6"
server_key_iv: "b90482d867b9b10c5cdea97293caa7c8"
server_key_version: "encryptionkey@0"
I20260806 18:34:39.373903 21129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "2e4bbef384f34b9ea41ce256fe9c08cc"
format_stamp: "Formatted at 2026-08-06 18:34:39 on dist-test-slave-8w9v"
server_key: "afea61f1183f73fe0d0dec4b18eab3a6"
server_key_iv: "b90482d867b9b10c5cdea97293caa7c8"
server_key_version: "encryptionkey@0"
I20260806 18:34:39.378268 21129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:39.381104 21146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:39.382506 21129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:39.382648 21129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "2e4bbef384f34b9ea41ce256fe9c08cc"
format_stamp: "Formatted at 2026-08-06 18:34:39 on dist-test-slave-8w9v"
server_key: "afea61f1183f73fe0d0dec4b18eab3a6"
server_key_iv: "b90482d867b9b10c5cdea97293caa7c8"
server_key_version: "encryptionkey@0"
I20260806 18:34:39.382773 21129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:39.417253 21129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:39.420634 21129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:39.420853 21129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:39.422469 21129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:39.422545 21129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:39.422619 21129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:39.422672 21129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:34:39 dist-test-slave-8w9v krb5kdc[20710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041279, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:39.433892 21129 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:40481
I20260806 18:34:39.433982 21269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:40481 every 8 connection(s)
I20260806 18:34:39.435016 21129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:39.437671 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.423444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:49713 (local address 127.18.125.190:42317)
0806 18:34:39.423576 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.423580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.424838 (+ 1258us) server_negotiation.cc:407] Connection header received
0806 18:34:39.426064 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.426068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.426156 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.426218 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:39.428021 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.428986 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.430451 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.430763 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.430881 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.434015 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:39.434020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:39.434022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:39.434056 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:39.435618 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.436272 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.436275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.436276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.436360 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.436673 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.436676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.436677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.436883 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:39.436964 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.437429 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.437541 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:34:39.438510 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.424095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42317 (local address 127.18.125.131:49713)
0806 18:34:39.424673 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.424718 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.425769 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.426648 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.426658 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.427069 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:39.427836 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.427856 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.429144 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.429146 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.430303 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.430314 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.430931 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.430935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.431122 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.431781 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:39.431806 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:39.433774 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:39.435759 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:39.435765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:39.435781 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:39.436105 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:39.436447 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:39.436449 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:39.436450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:39.436583 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:39.436980 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:39.436985 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:39.437215 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.437365 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.437625 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":136,"threads_started":1}
I20260806 18:34:39.439924 21270 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42317
I20260806 18:34:39.440143 21270 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:39.440696 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 21129
I20260806 18:34:39.440871 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:39.440879 21270 heartbeater.cc:511] Master 127.18.125.190:42317 requested a full tablet report, sending...
I20260806 18:34:39.441206 18934 external_mini_cluster.cc:1428] Setting key 85c04bdb321559d42727c66132c0998c
I20260806 18:34:39.441989 20757 ts_manager.cc:194] Registered new tserver with Master: 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481)
I20260806 18:34:39.442790 20757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:49713
I20260806 18:34:39.443244 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:39.451265 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.443290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42317 (local address 127.18.125.131:46749)
0806 18:34:39.443550 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.443562 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.443634 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.443929 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.443931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.444101 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:39.444295 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.444303 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.445387 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.445390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.446003 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.446012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.446629 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.446632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.446706 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.447230 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:39.447250 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:39.447602 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:39.449656 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:39.449659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:39.449662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:39.449968 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:39.450220 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:39.450226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:39.450231 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:39.450336 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:39.450682 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:39.450688 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:39.450812 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.450947 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.451102 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":98,"threads_started":1}
I20260806 18:34:39.451388 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.443607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:46749 (local address 127.18.125.190:42317)
0806 18:34:39.443714 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.443719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.443733 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:34:39.443772 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.443776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.443823 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.443928 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:39.444423 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.445240 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.446121 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.446516 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.446638 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.447748 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:39.447755 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:39.447761 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:39.447793 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:39.449542 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.450064 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.450067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.450068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.450137 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.450427 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.450429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.450431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.450593 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:39.450667 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.451025 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.451163 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 18:34:39 dist-test-slave-8w9v krb5kdc[20710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:34:39 dist-test-slave-8w9v krb5kdc[20710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:39.472002 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.463421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50876 (local address 127.18.125.190:42317)
0806 18:34:39.463548 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.463552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.463564 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:34:39.463605 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.463608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.463646 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.463715 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:39.464474 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.465009 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.465840 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.466067 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.466151 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.468641 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:39.468646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:39.468648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:39.468673 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:39.470236 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.470751 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.470760 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.470761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.470839 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.471113 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.471118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.471120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.471323 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:39.471463 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.471699 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.471820 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:34:39.475353 20757 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:39.478454 20757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:39.494596 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.487699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59008 (local address 127.18.125.131:40481)
0806 18:34:39.488781 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.488794 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.488830 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:34:39.488915 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.488920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.489145 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.489344 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:39.489666 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.490859 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.492690 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.493551 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.493717 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.493867 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.494328 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.494458 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":972,"thread_start_us":136,"threads_started":1}
I20260806 18:34:39.494587 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.488330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:38845 (local address 127.0.0.1:36746)
0806 18:34:39.488825 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.488840 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.488927 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.489607 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.489610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.489627 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:39.489748 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.489755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.490888 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.490895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.492574 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.492584 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.493713 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.493718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.493949 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.494051 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.494185 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.494366 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":269,"thread_start_us":98,"threads_started":1}
I20260806 18:34:39.494585 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.487535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:40481 (local address 127.0.0.1:59008)
0806 18:34:39.487914 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.487971 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.488109 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.489341 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.489347 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.489370 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:39.489522 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.489530 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.490962 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.490965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.492545 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.492557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.493713 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.493718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.493946 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.494047 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.494193 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.494371 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":180,"thread_start_us":105,"threads_started":1}
I20260806 18:34:39.495785 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.488772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36746 (local address 127.18.125.130:38845)
0806 18:34:39.489196 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.489207 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.489246 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:34:39.489335 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.489340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.489468 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.489612 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:39.489879 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.490776 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.492739 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.493551 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.493717 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.493867 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.495510 (+ 1643us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.495628 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":109,"threads_started":1}
I20260806 18:34:39.496096 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.487815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:32807 (local address 127.0.0.1:51928)
0806 18:34:39.488517 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.488533 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.488630 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.490086 (+ 1456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.490089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.490108 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:39.490211 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.490217 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.491466 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.491470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.492566 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.492573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.495617 (+ 3044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.495621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.495733 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.495758 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.495844 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.495930 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":60,"thread_start_us":102,"threads_started":1}
I20260806 18:34:39.496487 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.488307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51928 (local address 127.18.125.129:32807)
0806 18:34:39.489650 (+ 1343us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.489657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.489680 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:34:39.489755 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.489760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.489943 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.490092 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:39.490334 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.491341 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.493935 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.495493 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.495626 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.495753 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.496057 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.496354 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1219,"thread_start_us":107,"threads_started":1}
I20260806 18:34:39.498412 21049 tablet_service.cc:1467] Processing CreateTablet for tablet 3931e1d1ddbf457e9772202a4462d8d3 (DEFAULT_TABLE table=test-table [id=db51668de8c14fa6a9e18bb5638847ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:39.498459 21200 tablet_service.cc:1467] Processing CreateTablet for tablet 3931e1d1ddbf457e9772202a4462d8d3 (DEFAULT_TABLE table=test-table [id=db51668de8c14fa6a9e18bb5638847ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:39.499058 20905 tablet_service.cc:1467] Processing CreateTablet for tablet 3931e1d1ddbf457e9772202a4462d8d3 (DEFAULT_TABLE table=test-table [id=db51668de8c14fa6a9e18bb5638847ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:39.499636 21049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3931e1d1ddbf457e9772202a4462d8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:39.499657 21200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3931e1d1ddbf457e9772202a4462d8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:39.500025 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3931e1d1ddbf457e9772202a4462d8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:39.507153 21294 tablet_bootstrap.cc:492] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: Bootstrap starting.
I20260806 18:34:39.508337 21296 tablet_bootstrap.cc:492] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: Bootstrap starting.
I20260806 18:34:39.508579 21295 tablet_bootstrap.cc:492] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: Bootstrap starting.
I20260806 18:34:39.510345 21294 tablet_bootstrap.cc:654] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:39.510622 21296 tablet_bootstrap.cc:654] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:39.510622 21295 tablet_bootstrap.cc:654] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:39.511277 21294 log.cc:824] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:39.511546 21296 log.cc:824] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:39.511546 21295 log.cc:824] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:39.513468 21296 tablet_bootstrap.cc:492] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: No bootstrap required, opened a new log
I20260806 18:34:39.513568 21294 tablet_bootstrap.cc:492] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: No bootstrap required, opened a new log
I20260806 18:34:39.513731 21296 ts_tablet_manager.cc:1397] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:34:39.513758 21295 tablet_bootstrap.cc:492] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: No bootstrap required, opened a new log
I20260806 18:34:39.513772 21294 ts_tablet_manager.cc:1397] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260806 18:34:39.513919 21295 ts_tablet_manager.cc:1397] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:34:39.517599 21296 raft_consensus.cc:348] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } }
I20260806 18:34:39.517938 21296 raft_consensus.cc:374] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:39.517995 21296 raft_consensus.cc:729] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4bbef384f34b9ea41ce256fe9c08cc, State: Initialized, Role: FOLLOWER
I20260806 18:34:39.518033 21294 raft_consensus.cc:348] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } }
I20260806 18:34:39.518301 21294 raft_consensus.cc:374] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:39.518373 21294 raft_consensus.cc:729] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3e7949fb6e94c088283c87b4e716c25, State: Initialized, Role: FOLLOWER
I20260806 18:34:39.518541 21296 consensus_queue.cc:260] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } }
I20260806 18:34:39.518792 21294 consensus_queue.cc:260] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } }
I20260806 18:34:39.518754 21295 raft_consensus.cc:348] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } }
I20260806 18:34:39.519011 21295 raft_consensus.cc:374] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:39.519084 21295 raft_consensus.cc:729] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770f7edf17934c03a6f0fcf28c7943ec, State: Initialized, Role: FOLLOWER
I20260806 18:34:39.519574 21295 consensus_queue.cc:260] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } }
I20260806 18:34:39.519690 21270 heartbeater.cc:503] Master 127.18.125.190:42317 was elected leader, sending a full tablet report...
I20260806 18:34:39.519850 21296 ts_tablet_manager.cc:1428] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:34:39.519866 21294 ts_tablet_manager.cc:1428] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260806 18:34:39.520260 21120 heartbeater.cc:503] Master 127.18.125.190:42317 was elected leader, sending a full tablet report...
I20260806 18:34:39.520511 21295 ts_tablet_manager.cc:1428] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 18:34:39.520857 20971 heartbeater.cc:503] Master 127.18.125.190:42317 was elected leader, sending a full tablet report...
I20260806 18:34:39.636656 21303 raft_consensus.cc:482] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:39.636929 21303 raft_consensus.cc:504] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } }
I20260806 18:34:39.638164 21303 leader_election.cc:288] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481), 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807)
I20260806 18:34:39.642273 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.638561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:57523 (local address 127.18.125.129:32807)
0806 18:34:39.638753 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.638759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.638783 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:34:39.639006 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.639009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.639060 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.639145 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:39.639504 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.640103 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.641265 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.641703 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.641833 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.641889 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.642076 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.642151 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 18:34:39.642277 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.638498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:32807 (local address 127.18.125.130:57523)
0806 18:34:39.638720 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.638736 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.638838 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.639183 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.639186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.639215 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:39.639365 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.639375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.640270 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.640275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.641125 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.641134 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.641833 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.641837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.641947 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.641972 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.642059 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.642132 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 18:34:39.643054 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3931e1d1ddbf457e9772202a4462d8d3" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770f7edf17934c03a6f0fcf28c7943ec" is_pre_election: true
I20260806 18:34:39.643409 20925 raft_consensus.cc:2454] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 0.
I20260806 18:34:39.643522 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.638765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:50735 (local address 127.18.125.131:40481)
0806 18:34:39.638907 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.638914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.639219 (+ 305us) server_negotiation.cc:407] Connection header received
0806 18:34:39.639472 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.639475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.639534 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.639594 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:39.639961 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.640620 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.642331 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.642920 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.643058 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.643107 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.643333 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.643417 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:34:39.643535 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.638499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:40481 (local address 127.18.125.130:50735)
0806 18:34:39.639128 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.639143 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.639292 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.639711 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.639713 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.639726 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:39.639821 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.639825 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.640724 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.640731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.642205 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.642215 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.643060 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.643064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.643194 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.643228 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.643314 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.643381 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":304,"thread_start_us":81,"threads_started":1}
I20260806 18:34:39.643949 20998 leader_election.cc:302] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 770f7edf17934c03a6f0fcf28c7943ec, c3e7949fb6e94c088283c87b4e716c25; no voters:
I20260806 18:34:39.644300 21303 raft_consensus.cc:2781] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:39.644352 21216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3931e1d1ddbf457e9772202a4462d8d3" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" is_pre_election: true
I20260806 18:34:39.644415 21303 raft_consensus.cc:482] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:39.644465 21303 raft_consensus.cc:3037] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:39.644618 21216 raft_consensus.cc:2454] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 0.
I20260806 18:34:39.645612 21303 raft_consensus.cc:504] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } }
I20260806 18:34:39.645924 21303 leader_election.cc:288] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 election: Requested vote from peers 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481), 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807)
I20260806 18:34:39.646281 21216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3931e1d1ddbf457e9772202a4462d8d3" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc"
I20260806 18:34:39.646296 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3931e1d1ddbf457e9772202a4462d8d3" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770f7edf17934c03a6f0fcf28c7943ec"
I20260806 18:34:39.646430 20925 raft_consensus.cc:3037] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:39.646557 21216 raft_consensus.cc:3037] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:39.647504 20925 raft_consensus.cc:2454] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 1.
I20260806 18:34:39.647682 21216 raft_consensus.cc:2454] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 1.
I20260806 18:34:39.647855 20998 leader_election.cc:302] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 770f7edf17934c03a6f0fcf28c7943ec, c3e7949fb6e94c088283c87b4e716c25; no voters:
I20260806 18:34:39.648082 21303 raft_consensus.cc:2781] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:39.648371 21303 raft_consensus.cc:686] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 1 LEADER]: Becoming Leader. State: Replica: c3e7949fb6e94c088283c87b4e716c25, State: Running, Role: LEADER
I20260806 18:34:39.648710 21303 consensus_queue.cc:237] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } }
I20260806 18:34:39.651974 20758 catalog_manager.cc:5495] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3e7949fb6e94c088283c87b4e716c25 (127.18.125.130). New cstate: current_term: 1 leader_uuid: "c3e7949fb6e94c088283c87b4e716c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } health_report { overall_health: UNKNOWN } } }
W20260806 18:34:39.657944 20972 tablet.cc:2369] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:34:39.677016 21121 tablet.cc:2369] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:34:39.686388 21271 tablet.cc:2369] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 18:34:39 dist-test-slave-8w9v krb5kdc[20710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041278, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:39.711460 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.702147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42317 (local address 127.0.0.1:50892)
0806 18:34:39.702517 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.702531 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.702643 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.702926 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.702928 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.703328 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:39.703436 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.703442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.704250 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.704252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.704796 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.704803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.705206 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.705208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.705272 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.705819 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:39.705831 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:39.707292 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:39.709342 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:39.709352 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:39.709359 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:39.709630 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:39.710135 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:39.710138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:39.710139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:39.710231 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:39.710758 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:39.710776 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:39.710898 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.711024 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.711251 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":3,"thread_start_us":129,"threads_started":1}
I20260806 18:34:39.711459 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.702183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50892 (local address 127.18.125.190:42317)
0806 18:34:39.702330 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.702336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.702607 (+ 271us) server_negotiation.cc:407] Connection header received
0806 18:34:39.702760 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.702763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.702809 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.702890 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:39.703544 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.704092 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.704892 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.705105 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.705182 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.707430 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:39.707433 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:39.707435 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:39.707457 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:39.709212 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.709826 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.709832 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.709833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.709988 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:39.710356 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:39.710361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:39.710362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:39.710579 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:39.710757 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.711075 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.711330 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":49}
I20260806 18:34:39.716027 20758 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:50892:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:34:39.716864 20758 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:39.722481 20905 tablet_service.cc:1467] Processing CreateTablet for tablet b390096bea144ed6a3fb087f95c1adcb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d173553d81464e20ba97d57a8939238d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:39.722474 21049 tablet_service.cc:1467] Processing CreateTablet for tablet b390096bea144ed6a3fb087f95c1adcb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d173553d81464e20ba97d57a8939238d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:39.722949 21200 tablet_service.cc:1467] Processing CreateTablet for tablet b390096bea144ed6a3fb087f95c1adcb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d173553d81464e20ba97d57a8939238d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:34:39.723371 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b390096bea144ed6a3fb087f95c1adcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:39.723393 21049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b390096bea144ed6a3fb087f95c1adcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:39.723600 21200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b390096bea144ed6a3fb087f95c1adcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:39.726511 21295 tablet_bootstrap.cc:492] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec: Bootstrap starting.
I20260806 18:34:39.726593 21294 tablet_bootstrap.cc:492] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25: Bootstrap starting.
I20260806 18:34:39.726637 21296 tablet_bootstrap.cc:492] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: Bootstrap starting.
I20260806 18:34:39.727527 21295 tablet_bootstrap.cc:654] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:39.727631 21296 tablet_bootstrap.cc:654] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:39.727631 21294 tablet_bootstrap.cc:654] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:39.729148 21295 tablet_bootstrap.cc:492] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec: No bootstrap required, opened a new log
I20260806 18:34:39.729169 21296 tablet_bootstrap.cc:492] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: No bootstrap required, opened a new log
I20260806 18:34:39.729266 21295 ts_tablet_manager.cc:1397] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:39.729279 21296 ts_tablet_manager.cc:1397] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 18:34:39.729704 21296 raft_consensus.cc:348] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:39.729809 21296 raft_consensus.cc:374] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:39.729832 21296 raft_consensus.cc:729] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4bbef384f34b9ea41ce256fe9c08cc, State: Initialized, Role: FOLLOWER
I20260806 18:34:39.729737 21295 raft_consensus.cc:348] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:39.729955 21295 raft_consensus.cc:374] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:39.729969 21296 consensus_queue.cc:260] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:39.730039 21295 raft_consensus.cc:729] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770f7edf17934c03a6f0fcf28c7943ec, State: Initialized, Role: FOLLOWER
I20260806 18:34:39.730245 21296 ts_tablet_manager.cc:1428] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:34:39.730211 21295 consensus_queue.cc:260] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:39.730293 21302 tablet_replica.cc:439] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } }
I20260806 18:34:39.730464 21302 tablet_replica.cc:442] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:39.730616 21295 ts_tablet_manager.cc:1428] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:34:39.730576 21304 tablet_replica.cc:439] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } }
I20260806 18:34:39.730713 21304 tablet_replica.cc:442] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:39.731600 21294 tablet_bootstrap.cc:492] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25: No bootstrap required, opened a new log
I20260806 18:34:39.731719 21294 ts_tablet_manager.cc:1397] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 18:34:39.732282 21294 raft_consensus.cc:348] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:39.732419 21294 raft_consensus.cc:374] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:39.732455 21294 raft_consensus.cc:729] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3e7949fb6e94c088283c87b4e716c25, State: Initialized, Role: FOLLOWER
I20260806 18:34:39.732635 21294 consensus_queue.cc:260] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:39.732933 21294 ts_tablet_manager.cc:1428] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:39.732944 21303 tablet_replica.cc:439] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } }
I20260806 18:34:39.733124 21303 tablet_replica.cc:442] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:39.868109 21302 raft_consensus.cc:482] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:39.868330 21302 raft_consensus.cc:504] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:39.869690 21302 leader_election.cc:288] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807), c3e7949fb6e94c088283c87b4e716c25 (127.18.125.130:38845)
I20260806 18:34:39.874009 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.870172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:40421 (local address 127.18.125.129:32807)
0806 18:34:39.870340 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.870347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.870363 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:34:39.870420 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.870422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.870462 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.870551 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:39.870890 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.871535 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.872871 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.873325 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.873465 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.873528 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.873787 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.873881 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 18:34:39.874008 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.870097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:52525 (local address 127.18.125.130:38845)
0806 18:34:39.870253 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.870258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.870545 (+ 287us) server_negotiation.cc:407] Connection header received
0806 18:34:39.870724 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.870728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.870784 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.870872 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:39.871175 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.871768 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.872899 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.873315 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.873448 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.873510 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.873768 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.873864 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 18:34:39.874008 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.869904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:32807 (local address 127.18.125.131:40421)
0806 18:34:39.870125 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.870143 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.870255 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.870570 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.870572 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.870591 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:39.870740 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.870747 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.871665 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.871668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.872691 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.872702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.873468 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.873471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.873638 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.873660 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.873767 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.873885 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":117}
I20260806 18:34:39.874030 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.869968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:38845 (local address 127.18.125.131:52525)
0806 18:34:39.870444 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.870456 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.870592 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.870895 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.870898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.870916 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:39.871048 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.871055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.871883 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.871885 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.872747 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.872756 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.873460 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.873464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.873630 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.873659 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.873749 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.873859 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":39,"thread_start_us":136,"threads_started":1}
I20260806 18:34:39.874709 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b390096bea144ed6a3fb087f95c1adcb" candidate_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770f7edf17934c03a6f0fcf28c7943ec" is_pre_election: true
I20260806 18:34:39.874897 20925 raft_consensus.cc:2454] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e4bbef384f34b9ea41ce256fe9c08cc in term 0.
I20260806 18:34:39.875160 21069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b390096bea144ed6a3fb087f95c1adcb" candidate_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3e7949fb6e94c088283c87b4e716c25" is_pre_election: true
I20260806 18:34:39.875341 21148 leader_election.cc:302] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e4bbef384f34b9ea41ce256fe9c08cc, 770f7edf17934c03a6f0fcf28c7943ec; no voters:
I20260806 18:34:39.875422 21069 raft_consensus.cc:2454] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e4bbef384f34b9ea41ce256fe9c08cc in term 0.
I20260806 18:34:39.875667 21302 raft_consensus.cc:2781] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:39.875744 21302 raft_consensus.cc:482] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:39.875799 21302 raft_consensus.cc:3037] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:39.877080 21302 raft_consensus.cc:504] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:39.877497 21302 leader_election.cc:288] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [CANDIDATE]: Term 1 election: Requested vote from peers 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807), c3e7949fb6e94c088283c87b4e716c25 (127.18.125.130:38845)
I20260806 18:34:39.877887 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b390096bea144ed6a3fb087f95c1adcb" candidate_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770f7edf17934c03a6f0fcf28c7943ec"
I20260806 18:34:39.877956 21069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b390096bea144ed6a3fb087f95c1adcb" candidate_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3e7949fb6e94c088283c87b4e716c25"
I20260806 18:34:39.878046 20925 raft_consensus.cc:3037] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:39.878095 21069 raft_consensus.cc:3037] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:39.879068 20925 raft_consensus.cc:2454] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e4bbef384f34b9ea41ce256fe9c08cc in term 1.
I20260806 18:34:39.879122 21069 raft_consensus.cc:2454] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e4bbef384f34b9ea41ce256fe9c08cc in term 1.
I20260806 18:34:39.879396 21148 leader_election.cc:302] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e4bbef384f34b9ea41ce256fe9c08cc, 770f7edf17934c03a6f0fcf28c7943ec; no voters:
I20260806 18:34:39.879578 21302 raft_consensus.cc:2781] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:39.879796 21302 raft_consensus.cc:686] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 LEADER]: Becoming Leader. State: Replica: 2e4bbef384f34b9ea41ce256fe9c08cc, State: Running, Role: LEADER
I20260806 18:34:39.880306 21302 consensus_queue.cc:237] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:39.882769 21318 tablet_replica.cc:439] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: TxnStatusTablet state changed. Reason: New leader 2e4bbef384f34b9ea41ce256fe9c08cc. Latest consensus state: current_term: 1 leader_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } }
I20260806 18:34:39.882899 21318 tablet_replica.cc:442] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:39.883201 21320 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:34:39.883430 21320 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:34:39.883812 20756 catalog_manager.cc:5495] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131). New cstate: current_term: 1 leader_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } health_report { overall_health: HEALTHY } } }
I20260806 18:34:39.923539 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.919782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59012 (local address 127.18.125.131:40481)
0806 18:34:39.919953 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.919960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.919979 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:34:39.920066 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.920068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.920166 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.920268 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:39.920519 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.921024 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.922009 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.922226 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.922331 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.922627 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:39.923138 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:39.923257 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.923330 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.923385 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:34:39.923539 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.919607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:40481 (local address 127.0.0.1:59012)
0806 18:34:39.919763 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.919779 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.919901 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.920295 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.920298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.920306 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:39.920421 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.920429 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.921131 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.921136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.921875 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.921884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.922333 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.922337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.922430 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.922476 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:39.923232 (+ 756us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:39.923249 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.923320 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.923414 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 18:34:39.929486 21069 raft_consensus.cc:1261] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Refusing update from remote peer 2e4bbef384f34b9ea41ce256fe9c08cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:39.929556 20925 raft_consensus.cc:1261] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Refusing update from remote peer 2e4bbef384f34b9ea41ce256fe9c08cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:39.930219 21302 consensus_queue.cc:1035] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:39.930694 21318 consensus_queue.cc:1035] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:39.933974 21303 tablet_replica.cc:439] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25: TxnStatusTablet state changed. Reason: New leader 2e4bbef384f34b9ea41ce256fe9c08cc. Latest consensus state: current_term: 1 leader_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } }
I20260806 18:34:39.934105 21303 tablet_replica.cc:442] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:39.934687 21304 tablet_replica.cc:439] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec: TxnStatusTablet state changed. Reason: New leader 2e4bbef384f34b9ea41ce256fe9c08cc. Latest consensus state: current_term: 1 leader_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } }
I20260806 18:34:39.934808 21304 tablet_replica.cc:442] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:39.938170 21318 tablet_replica.cc:439] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } }
I20260806 18:34:39.938297 21318 tablet_replica.cc:442] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:39.938426 21318 tablet_replica.cc:439] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } }
I20260806 18:34:39.938501 21318 tablet_replica.cc:442] T b390096bea144ed6a3fb087f95c1adcb P 2e4bbef384f34b9ea41ce256fe9c08cc: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:34:39.939013 21303 tablet_replica.cc:439] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } }
I20260806 18:34:39.939141 21303 tablet_replica.cc:442] T b390096bea144ed6a3fb087f95c1adcb P c3e7949fb6e94c088283c87b4e716c25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:39.939417 21324 mvcc.cc:205] Tried to move back new op lower bound from 7315625082583801856 to 7315625082392989696. Current Snapshot: MvccSnapshot[applied={T|T < 7315625082583801856}]
I20260806 18:34:39.939575 21325 mvcc.cc:205] Tried to move back new op lower bound from 7315625082583801856 to 7315625082392989696. Current Snapshot: MvccSnapshot[applied={T|T < 7315625082583801856}]
I20260806 18:34:39.941366 21304 tablet_replica.cc:439] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } } }
I20260806 18:34:39.941524 21304 tablet_replica.cc:442] T b390096bea144ed6a3fb087f95c1adcb P 770f7edf17934c03a6f0fcf28c7943ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:34:39.943730 21326 mvcc.cc:205] Tried to move back new op lower bound from 7315625082583801856 to 7315625082392989696. Current Snapshot: MvccSnapshot[applied={T|T < 7315625082583801856}]
I20260806 18:34:39.952066 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.947402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36750 (local address 127.18.125.130:38845)
0806 18:34:39.947539 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.947543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.947560 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:34:39.947739 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.947742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.947804 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.947906 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:39.948258 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.948816 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.950174 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.950449 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.950550 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.950932 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:39.951553 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:39.951685 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.951783 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.951846 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:34:39.966277 21336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.962601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:40481 (local address 127.18.125.130:51025)
0806 18:34:39.962975 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.962988 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.963134 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.963411 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.963414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.963422 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:39.963553 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.963558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.964352 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.964356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.965000 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.965009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.965466 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.965470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.965566 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.965614 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:39.965946 (+ 332us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:39.965955 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 18:34:39.966035 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 18:34:39.966117 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":123,"threads_started":1}
I20260806 18:34:39.966256 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.962718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:51025 (local address 127.18.125.131:40481)
0806 18:34:39.962877 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.962882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.963066 (+ 184us) server_negotiation.cc:407] Connection header received
0806 18:34:39.963247 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.963251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.963306 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.963386 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:39.963660 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.964246 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.965127 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.965350 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.965456 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.965787 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:39.965864 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:39.965941 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:39.966049 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 18:34:39.966102 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 18:34:39.974210 20925 raft_consensus.cc:1261] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Refusing update from remote peer c3e7949fb6e94c088283c87b4e716c25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:39.974330 21216 raft_consensus.cc:1261] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Refusing update from remote peer c3e7949fb6e94c088283c87b4e716c25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:39.974828 21303 consensus_queue.cc:1035] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:39.975028 21306 consensus_queue.cc:1035] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:40.000707 21340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.996451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:38845 (local address 127.18.125.131:32849)
0806 18:34:39.996831 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:39.996848 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:39.996998 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:39.997259 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:39.997261 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:39.997270 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:34:39.997384 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.997390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.998319 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.998322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.999269 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:39.999282 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.999751 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.999755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:39.999846 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:39.999905 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:34:40.000279 (+ 374us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:34:40.000289 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 18:34:40.000426 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 18:34:40.000539 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":159,"threads_started":1}
I20260806 18:34:40.000700 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:39.996838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:32849 (local address 127.18.125.130:38845)
0806 18:34:39.996980 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:39.996985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:39.997000 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:34:39.997095 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:39.997098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:39.997152 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:39.997230 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:39.997532 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.998217 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.999406 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:39.999626 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:39.999711 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:40.000066 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:40.000192 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:40.000268 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:40.000473 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 18:34:40.000548 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:34:40.031991 20756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:40.032261 20756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:40.035650 20756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822: Sending DeleteTablet for 3 replicas of tablet 3931e1d1ddbf457e9772202a4462d8d3
I20260806 18:34:40.036763 21200 tablet_service.cc:1514] Processing DeleteTablet for tablet 3931e1d1ddbf457e9772202a4462d8d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:40 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:59008
I20260806 18:34:40.036846 20905 tablet_service.cc:1514] Processing DeleteTablet for tablet 3931e1d1ddbf457e9772202a4462d8d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:40 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:51928
I20260806 18:34:40.037652 21049 tablet_service.cc:1514] Processing DeleteTablet for tablet 3931e1d1ddbf457e9772202a4462d8d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:40 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:36746
I20260806 18:34:40.038262 20756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:40.038781 20756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:40.038823 21346 tablet_replica.cc:330] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: stopping tablet replica
I20260806 18:34:40.039361 21346 raft_consensus.cc:2229] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:40.039633 21344 tablet_replica.cc:330] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: stopping tablet replica
I20260806 18:34:40.040390 21346 raft_consensus.cc:2258] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:40.040796 21345 tablet_replica.cc:330] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: stopping tablet replica
I20260806 18:34:40.041100 21345 raft_consensus.cc:2229] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:40.041327 21345 raft_consensus.cc:2258] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:40.043237 21344 raft_consensus.cc:2229] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:40.043502 21344 raft_consensus.cc:2258] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:40.045503 21344 ts_tablet_manager.cc:1905] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:40.046192 21049 tablet_service.cc:1467] Processing CreateTablet for tablet 7e525e27011c46efad9444f7429c468e (DEFAULT_TABLE table=test-table [id=933f8dbc8bb749f1b7cef1a42479862d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:40.046545 21049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e525e27011c46efad9444f7429c468e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:40.046800 20905 tablet_service.cc:1467] Processing CreateTablet for tablet 7e525e27011c46efad9444f7429c468e (DEFAULT_TABLE table=test-table [id=933f8dbc8bb749f1b7cef1a42479862d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:40.047102 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e525e27011c46efad9444f7429c468e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:40.049780 21295 tablet_bootstrap.cc:492] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec: Bootstrap starting.
I20260806 18:34:40.050215 21294 tablet_bootstrap.cc:492] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25: Bootstrap starting.
I20260806 18:34:40.050828 21295 tablet_bootstrap.cc:654] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:40.051278 21294 tablet_bootstrap.cc:654] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:40.053210 21345 ts_tablet_manager.cc:1905] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:40.053604 21346 ts_tablet_manager.cc:1905] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:40.056866 21346 ts_tablet_manager.cc:1918] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:40.049755 21200 tablet_service.cc:1467] Processing CreateTablet for tablet 7e525e27011c46efad9444f7429c468e (DEFAULT_TABLE table=test-table [id=933f8dbc8bb749f1b7cef1a42479862d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:40.057320 21200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e525e27011c46efad9444f7429c468e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:40.058589 21345 ts_tablet_manager.cc:1918] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:40.058698 21345 log.cc:1196] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/wal/wals/3931e1d1ddbf457e9772202a4462d8d3
I20260806 18:34:40.059051 21345 ts_tablet_manager.cc:1939] T 3931e1d1ddbf457e9772202a4462d8d3 P 770f7edf17934c03a6f0fcf28c7943ec: Deleting consensus metadata
I20260806 18:34:40.060035 21296 tablet_bootstrap.cc:492] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc: Bootstrap starting.
I20260806 18:34:40.060549 20743 catalog_manager.cc:4841] TS 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807): tablet 3931e1d1ddbf457e9772202a4462d8d3 (table test-table [id=db51668de8c14fa6a9e18bb5638847ee]) successfully deleted
I20260806 18:34:40.061147 21296 tablet_bootstrap.cc:654] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:40.066259 21296 tablet_bootstrap.cc:492] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc: No bootstrap required, opened a new log
I20260806 18:34:40.066366 21296 ts_tablet_manager.cc:1397] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260806 18:34:40.066576 21295 tablet_bootstrap.cc:492] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec: No bootstrap required, opened a new log
I20260806 18:34:40.066665 21295 ts_tablet_manager.cc:1397] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260806 18:34:40.066847 21296 raft_consensus.cc:348] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:40.066967 21296 raft_consensus.cc:374] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:40.066996 21296 raft_consensus.cc:729] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4bbef384f34b9ea41ce256fe9c08cc, State: Initialized, Role: FOLLOWER
I20260806 18:34:40.067126 21296 consensus_queue.cc:260] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:40.067119 21295 raft_consensus.cc:348] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:40.067233 21295 raft_consensus.cc:374] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:40.067263 21295 raft_consensus.cc:729] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770f7edf17934c03a6f0fcf28c7943ec, State: Initialized, Role: FOLLOWER
I20260806 18:34:40.067378 21296 ts_tablet_manager.cc:1428] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:34:40.067389 21295 consensus_queue.cc:260] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:40.067688 21295 ts_tablet_manager.cc:1428] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:40.067955 21346 log.cc:1196] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/3931e1d1ddbf457e9772202a4462d8d3
I20260806 18:34:40.065110 21294 tablet_bootstrap.cc:492] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25: No bootstrap required, opened a new log
I20260806 18:34:40.068359 21294 ts_tablet_manager.cc:1397] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20260806 18:34:40.068408 21346 ts_tablet_manager.cc:1939] T 3931e1d1ddbf457e9772202a4462d8d3 P c3e7949fb6e94c088283c87b4e716c25: Deleting consensus metadata
I20260806 18:34:40.068856 21294 raft_consensus.cc:348] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:40.068987 21294 raft_consensus.cc:374] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:40.069032 21294 raft_consensus.cc:729] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3e7949fb6e94c088283c87b4e716c25, State: Initialized, Role: FOLLOWER
I20260806 18:34:40.069188 21294 consensus_queue.cc:260] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:40.069461 21294 ts_tablet_manager.cc:1428] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:40.071486 20742 catalog_manager.cc:4841] TS c3e7949fb6e94c088283c87b4e716c25 (127.18.125.130:38845): tablet 3931e1d1ddbf457e9772202a4462d8d3 (table test-table [id=db51668de8c14fa6a9e18bb5638847ee]) successfully deleted
I20260806 18:34:40.073633 21344 ts_tablet_manager.cc:1918] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:40.073835 21344 log.cc:1196] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/3931e1d1ddbf457e9772202a4462d8d3
I20260806 18:34:40.074244 21344 ts_tablet_manager.cc:1939] T 3931e1d1ddbf457e9772202a4462d8d3 P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting consensus metadata
I20260806 18:34:40.075775 20742 catalog_manager.cc:4841] TS 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481): tablet 3931e1d1ddbf457e9772202a4462d8d3 (table test-table [id=db51668de8c14fa6a9e18bb5638847ee]) successfully deleted
I20260806 18:34:40.131559 21306 raft_consensus.cc:482] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:40.131731 21306 raft_consensus.cc:504] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:40.132781 21306 leader_election.cc:288] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807), 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481)
I20260806 18:34:40.133503 20923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e525e27011c46efad9444f7429c468e" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770f7edf17934c03a6f0fcf28c7943ec" is_pre_election: true
I20260806 18:34:40.133563 21216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e525e27011c46efad9444f7429c468e" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" is_pre_election: true
I20260806 18:34:40.133695 20923 raft_consensus.cc:2454] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 0.
I20260806 18:34:40.133733 21216 raft_consensus.cc:2454] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 0.
I20260806 18:34:40.134037 20998 leader_election.cc:302] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 770f7edf17934c03a6f0fcf28c7943ec, c3e7949fb6e94c088283c87b4e716c25; no voters:
I20260806 18:34:40.134222 21306 raft_consensus.cc:2781] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:40.134297 21306 raft_consensus.cc:482] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:40.134354 21306 raft_consensus.cc:3037] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:40.135418 21306 raft_consensus.cc:504] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:40.135849 21306 leader_election.cc:288] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 election: Requested vote from peers 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807), 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481)
I20260806 18:34:40.136312 21216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e525e27011c46efad9444f7429c468e" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc"
I20260806 18:34:40.136302 20923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e525e27011c46efad9444f7429c468e" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770f7edf17934c03a6f0fcf28c7943ec"
I20260806 18:34:40.136457 20923 raft_consensus.cc:3037] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:40.136461 21216 raft_consensus.cc:3037] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:40.137475 21216 raft_consensus.cc:2454] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 1.
I20260806 18:34:40.137748 20923 raft_consensus.cc:2454] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 1.
I20260806 18:34:40.137821 20997 leader_election.cc:302] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e4bbef384f34b9ea41ce256fe9c08cc, c3e7949fb6e94c088283c87b4e716c25; no voters:
I20260806 18:34:40.138024 21306 raft_consensus.cc:2781] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:40.138208 21306 raft_consensus.cc:686] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 1 LEADER]: Becoming Leader. State: Replica: c3e7949fb6e94c088283c87b4e716c25, State: Running, Role: LEADER
I20260806 18:34:40.138386 21306 consensus_queue.cc:237] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:40.140601 20757 catalog_manager.cc:5495] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3e7949fb6e94c088283c87b4e716c25 (127.18.125.130). New cstate: current_term: 1 leader_uuid: "c3e7949fb6e94c088283c87b4e716c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:40.168393 21216 raft_consensus.cc:1261] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Refusing update from remote peer c3e7949fb6e94c088283c87b4e716c25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:40.168931 21342 consensus_queue.cc:1035] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:40.169675 20923 raft_consensus.cc:1261] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Refusing update from remote peer c3e7949fb6e94c088283c87b4e716c25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:40.170674 21342 consensus_queue.cc:1035] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:34:40.191731 21325 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 18:34:41.759850 20756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:41.760205 20756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:41.764036 20756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822: Sending DeleteTablet for 3 replicas of tablet 7e525e27011c46efad9444f7429c468e
I20260806 18:34:41.764915 20905 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e525e27011c46efad9444f7429c468e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:41 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:51928
I20260806 18:34:41.765103 21200 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e525e27011c46efad9444f7429c468e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:41 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:59008
I20260806 18:34:41.765125 21049 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e525e27011c46efad9444f7429c468e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:41 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:36746
I20260806 18:34:41.765503 21377 tablet_replica.cc:330] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc: stopping tablet replica
I20260806 18:34:41.765672 21377 raft_consensus.cc:2229] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:41.765830 21377 raft_consensus.cc:2258] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:41.765965 21378 tablet_replica.cc:330] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25: stopping tablet replica
I20260806 18:34:41.766114 21378 raft_consensus.cc:2229] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:41.766343 21378 raft_consensus.cc:2258] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:41.766644 21377 ts_tablet_manager.cc:1905] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:41.766993 21378 ts_tablet_manager.cc:1905] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:41.769748 21378 ts_tablet_manager.cc:1918] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:34:41.769824 21378 log.cc:1196] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/7e525e27011c46efad9444f7429c468e
I20260806 18:34:41.769904 21377 ts_tablet_manager.cc:1918] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:34:41.770015 21377 log.cc:1196] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/7e525e27011c46efad9444f7429c468e
I20260806 18:34:41.770164 21378 ts_tablet_manager.cc:1939] T 7e525e27011c46efad9444f7429c468e P c3e7949fb6e94c088283c87b4e716c25: Deleting consensus metadata
I20260806 18:34:41.770298 21377 ts_tablet_manager.cc:1939] T 7e525e27011c46efad9444f7429c468e P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting consensus metadata
I20260806 18:34:41.771001 21376 tablet_replica.cc:330] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec: stopping tablet replica
I20260806 18:34:41.771008 20742 catalog_manager.cc:4841] TS c3e7949fb6e94c088283c87b4e716c25 (127.18.125.130:38845): tablet 7e525e27011c46efad9444f7429c468e (table test-table [id=933f8dbc8bb749f1b7cef1a42479862d]) successfully deleted
I20260806 18:34:41.771171 21376 raft_consensus.cc:2229] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:41.771224 20742 catalog_manager.cc:4841] TS 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481): tablet 7e525e27011c46efad9444f7429c468e (table test-table [id=933f8dbc8bb749f1b7cef1a42479862d]) successfully deleted
I20260806 18:34:41.771312 21376 raft_consensus.cc:2258] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:41.772225 21376 ts_tablet_manager.cc:1905] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:41.775875 21376 ts_tablet_manager.cc:1918] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:34:41.776010 21376 log.cc:1196] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/wal/wals/7e525e27011c46efad9444f7429c468e
I20260806 18:34:41.776366 21376 ts_tablet_manager.cc:1939] T 7e525e27011c46efad9444f7429c468e P 770f7edf17934c03a6f0fcf28c7943ec: Deleting consensus metadata
I20260806 18:34:41.777213 20743 catalog_manager.cc:4841] TS 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807): tablet 7e525e27011c46efad9444f7429c468e (table test-table [id=933f8dbc8bb749f1b7cef1a42479862d]) successfully deleted
Aug 06 18:34:41 dist-test-slave-8w9v krb5kdc[20710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041281, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:34:41 dist-test-slave-8w9v krb5kdc[20710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041281, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:41.798866 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:41.789763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50898 (local address 127.18.125.190:42317)
0806 18:34:41.790030 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:41.790037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:41.790073 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:34:41.790188 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:41.790191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:41.790239 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:41.790342 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:41.791038 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:41.791613 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:41.792527 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:41.792770 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:41.792850 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:41.795301 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:41.795304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:41.795306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:41.795379 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:41.797293 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:41.797720 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:41.797723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:41.797724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:41.797801 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:41.798035 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:41.798037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:41.798038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:41.798220 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:41.798342 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:41.798613 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 18:34:41.798721 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":110,"threads_started":1}
I20260806 18:34:41.801826 20756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:41.802371 20756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:41.807569 21200 tablet_service.cc:1467] Processing CreateTablet for tablet 7c3278c918994f0880080d01a4706fa9 (DEFAULT_TABLE table=test-table [id=6cc4dada86d34394b1ec37aeb4c7af01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:41.807567 20905 tablet_service.cc:1467] Processing CreateTablet for tablet 7c3278c918994f0880080d01a4706fa9 (DEFAULT_TABLE table=test-table [id=6cc4dada86d34394b1ec37aeb4c7af01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:41.807582 21049 tablet_service.cc:1467] Processing CreateTablet for tablet 7c3278c918994f0880080d01a4706fa9 (DEFAULT_TABLE table=test-table [id=6cc4dada86d34394b1ec37aeb4c7af01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:41.807888 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c3278c918994f0880080d01a4706fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:41.807888 21200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c3278c918994f0880080d01a4706fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:41.807896 21049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c3278c918994f0880080d01a4706fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:41.810652 21389 tablet_bootstrap.cc:492] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc: Bootstrap starting.
I20260806 18:34:41.810650 21388 tablet_bootstrap.cc:492] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25: Bootstrap starting.
I20260806 18:34:41.810756 21390 tablet_bootstrap.cc:492] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec: Bootstrap starting.
I20260806 18:34:41.811606 21388 tablet_bootstrap.cc:654] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:41.811606 21389 tablet_bootstrap.cc:654] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:41.811630 21390 tablet_bootstrap.cc:654] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:41.813105 21389 tablet_bootstrap.cc:492] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc: No bootstrap required, opened a new log
I20260806 18:34:41.813215 21389 ts_tablet_manager.cc:1397] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:41.813766 21388 tablet_bootstrap.cc:492] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25: No bootstrap required, opened a new log
I20260806 18:34:41.813803 21390 tablet_bootstrap.cc:492] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec: No bootstrap required, opened a new log
I20260806 18:34:41.813720 21389 raft_consensus.cc:348] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.813894 21389 raft_consensus.cc:374] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:41.813905 21390 ts_tablet_manager.cc:1397] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:41.813941 21389 raft_consensus.cc:729] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4bbef384f34b9ea41ce256fe9c08cc, State: Initialized, Role: FOLLOWER
I20260806 18:34:41.813972 21388 ts_tablet_manager.cc:1397] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:41.814100 21389 consensus_queue.cc:260] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.814426 21390 raft_consensus.cc:348] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.814484 21389 ts_tablet_manager.cc:1428] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:34:41.814543 21390 raft_consensus.cc:374] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:41.814580 21390 raft_consensus.cc:729] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770f7edf17934c03a6f0fcf28c7943ec, State: Initialized, Role: FOLLOWER
I20260806 18:34:41.814522 21388 raft_consensus.cc:348] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.814631 21388 raft_consensus.cc:374] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:41.814669 21388 raft_consensus.cc:729] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3e7949fb6e94c088283c87b4e716c25, State: Initialized, Role: FOLLOWER
I20260806 18:34:41.814699 21390 consensus_queue.cc:260] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.814811 21388 consensus_queue.cc:260] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.815075 21390 ts_tablet_manager.cc:1428] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:41.816324 21388 ts_tablet_manager.cc:1428] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:41.832633 21371 raft_consensus.cc:482] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:41.832770 21371 raft_consensus.cc:504] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.833211 21371 leader_election.cc:288] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807), 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481)
I20260806 18:34:41.833642 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c3278c918994f0880080d01a4706fa9" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770f7edf17934c03a6f0fcf28c7943ec" is_pre_election: true
I20260806 18:34:41.833691 21216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c3278c918994f0880080d01a4706fa9" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" is_pre_election: true
I20260806 18:34:41.833814 20925 raft_consensus.cc:2454] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 0.
I20260806 18:34:41.833846 21216 raft_consensus.cc:2454] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 0.
I20260806 18:34:41.834175 20998 leader_election.cc:302] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2e4bbef384f34b9ea41ce256fe9c08cc, 770f7edf17934c03a6f0fcf28c7943ec, c3e7949fb6e94c088283c87b4e716c25; no voters:
I20260806 18:34:41.834386 21371 raft_consensus.cc:2781] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:41.834463 21371 raft_consensus.cc:482] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:41.834517 21371 raft_consensus.cc:3037] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:41.835593 21371 raft_consensus.cc:504] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.835968 21371 leader_election.cc:288] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 election: Requested vote from peers 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807), 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481)
I20260806 18:34:41.836422 20925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c3278c918994f0880080d01a4706fa9" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770f7edf17934c03a6f0fcf28c7943ec"
I20260806 18:34:41.836438 21216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c3278c918994f0880080d01a4706fa9" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc"
I20260806 18:34:41.836575 21216 raft_consensus.cc:3037] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:41.836580 20925 raft_consensus.cc:3037] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:41.837807 20925 raft_consensus.cc:2454] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 1.
I20260806 18:34:41.837807 21216 raft_consensus.cc:2454] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 1.
I20260806 18:34:41.838176 20997 leader_election.cc:302] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2e4bbef384f34b9ea41ce256fe9c08cc, 770f7edf17934c03a6f0fcf28c7943ec, c3e7949fb6e94c088283c87b4e716c25; no voters:
I20260806 18:34:41.838349 21371 raft_consensus.cc:2781] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:41.838439 21371 raft_consensus.cc:686] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 1 LEADER]: Becoming Leader. State: Replica: c3e7949fb6e94c088283c87b4e716c25, State: Running, Role: LEADER
I20260806 18:34:41.838584 21371 consensus_queue.cc:237] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.840474 20756 catalog_manager.cc:5495] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3e7949fb6e94c088283c87b4e716c25 (127.18.125.130). New cstate: current_term: 1 leader_uuid: "c3e7949fb6e94c088283c87b4e716c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:41.860817 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:41.857452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36766 (local address 127.18.125.130:38845)
0806 18:34:41.857745 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:41.857759 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:41.857772 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:34:41.857824 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:41.857827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:41.857869 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:41.857959 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:41.858219 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:41.858771 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:41.859659 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:41.859899 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:41.860008 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:41.860329 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:41.860407 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:41.860487 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:41.860585 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 18:34:41.860644 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
I20260806 18:34:41.867787 21216 raft_consensus.cc:1261] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Refusing update from remote peer c3e7949fb6e94c088283c87b4e716c25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:41.867856 20925 raft_consensus.cc:1261] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Refusing update from remote peer c3e7949fb6e94c088283c87b4e716c25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:41.868247 21371 consensus_queue.cc:1035] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:41.868422 21342 consensus_queue.cc:1035] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:41.903762 20756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:41.903957 20756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:41.906083 20756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822: Sending DeleteTablet for 3 replicas of tablet 7c3278c918994f0880080d01a4706fa9
I20260806 18:34:41.906651 20905 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c3278c918994f0880080d01a4706fa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:41 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:51928
I20260806 18:34:41.906688 21049 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c3278c918994f0880080d01a4706fa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:41 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:36746
I20260806 18:34:41.906824 21376 tablet_replica.cc:330] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec: stopping tablet replica
I20260806 18:34:41.906889 21378 tablet_replica.cc:330] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25: stopping tablet replica
I20260806 18:34:41.906956 21376 raft_consensus.cc:2229] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:41.907017 21378 raft_consensus.cc:2229] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:41.907081 21376 raft_consensus.cc:2258] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:41.907218 21378 raft_consensus.cc:2258] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:41.907680 21376 ts_tablet_manager.cc:1905] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:41.907896 21378 ts_tablet_manager.cc:1905] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:41.908031 21200 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c3278c918994f0880080d01a4706fa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:41 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:59008
I20260806 18:34:41.908241 21377 tablet_replica.cc:330] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc: stopping tablet replica
I20260806 18:34:41.908367 21377 raft_consensus.cc:2229] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:41.908490 21377 raft_consensus.cc:2258] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:41.909186 20756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:41.909789 20756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:41.910483 21376 ts_tablet_manager.cc:1918] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:41.910566 21376 log.cc:1196] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-0/wal/wals/7c3278c918994f0880080d01a4706fa9
I20260806 18:34:41.910712 21378 ts_tablet_manager.cc:1918] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:41.910782 21378 log.cc:1196] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/7c3278c918994f0880080d01a4706fa9
I20260806 18:34:41.910826 21376 ts_tablet_manager.cc:1939] T 7c3278c918994f0880080d01a4706fa9 P 770f7edf17934c03a6f0fcf28c7943ec: Deleting consensus metadata
I20260806 18:34:41.911054 21378 ts_tablet_manager.cc:1939] T 7c3278c918994f0880080d01a4706fa9 P c3e7949fb6e94c088283c87b4e716c25: Deleting consensus metadata
I20260806 18:34:41.911813 21377 ts_tablet_manager.cc:1905] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:41.912029 20743 catalog_manager.cc:4841] TS 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807): tablet 7c3278c918994f0880080d01a4706fa9 (table test-table [id=6cc4dada86d34394b1ec37aeb4c7af01]) successfully deleted
I20260806 18:34:41.912431 20742 catalog_manager.cc:4841] TS c3e7949fb6e94c088283c87b4e716c25 (127.18.125.130:38845): tablet 7c3278c918994f0880080d01a4706fa9 (table test-table [id=6cc4dada86d34394b1ec37aeb4c7af01]) successfully deleted
I20260806 18:34:41.914675 21377 ts_tablet_manager.cc:1918] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:34:41.914755 21377 log.cc:1196] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/7c3278c918994f0880080d01a4706fa9
I20260806 18:34:41.915043 21377 ts_tablet_manager.cc:1939] T 7c3278c918994f0880080d01a4706fa9 P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting consensus metadata
I20260806 18:34:41.915226 20905 tablet_service.cc:1467] Processing CreateTablet for tablet 5d14a212704246fa8069b1a573db7272 (DEFAULT_TABLE table=test-table [id=69b0205065174a8487dc2bb34fbbdabc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:41.915504 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d14a212704246fa8069b1a573db7272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:41.916049 21049 tablet_service.cc:1467] Processing CreateTablet for tablet 5d14a212704246fa8069b1a573db7272 (DEFAULT_TABLE table=test-table [id=69b0205065174a8487dc2bb34fbbdabc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:41.916373 21049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d14a212704246fa8069b1a573db7272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:41.917999 21390 tablet_bootstrap.cc:492] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec: Bootstrap starting.
I20260806 18:34:41.918641 21388 tablet_bootstrap.cc:492] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25: Bootstrap starting.
I20260806 18:34:41.918967 21200 tablet_service.cc:1467] Processing CreateTablet for tablet 5d14a212704246fa8069b1a573db7272 (DEFAULT_TABLE table=test-table [id=69b0205065174a8487dc2bb34fbbdabc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:41.919091 21390 tablet_bootstrap.cc:654] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:41.919251 21200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d14a212704246fa8069b1a573db7272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:41.919574 21388 tablet_bootstrap.cc:654] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:41.920867 21390 tablet_bootstrap.cc:492] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec: No bootstrap required, opened a new log
I20260806 18:34:41.920948 21390 ts_tablet_manager.cc:1397] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:41.921376 21390 raft_consensus.cc:348] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.921476 21390 raft_consensus.cc:374] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:41.921499 21390 raft_consensus.cc:729] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770f7edf17934c03a6f0fcf28c7943ec, State: Initialized, Role: FOLLOWER
I20260806 18:34:41.921597 21389 tablet_bootstrap.cc:492] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc: Bootstrap starting.
I20260806 18:34:41.921651 21390 consensus_queue.cc:260] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.921916 21390 ts_tablet_manager.cc:1428] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:41.922433 21389 tablet_bootstrap.cc:654] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:41.922843 21388 tablet_bootstrap.cc:492] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25: No bootstrap required, opened a new log
I20260806 18:34:41.922963 21388 ts_tablet_manager.cc:1397] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 18:34:41.923173 20742 catalog_manager.cc:4841] TS 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481): tablet 7c3278c918994f0880080d01a4706fa9 (table test-table [id=6cc4dada86d34394b1ec37aeb4c7af01]) successfully deleted
I20260806 18:34:41.923331 21388 raft_consensus.cc:348] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.923453 21388 raft_consensus.cc:374] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:41.923481 21388 raft_consensus.cc:729] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3e7949fb6e94c088283c87b4e716c25, State: Initialized, Role: FOLLOWER
I20260806 18:34:41.923593 21388 consensus_queue.cc:260] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.923832 21388 ts_tablet_manager.cc:1428] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:41.924664 21389 tablet_bootstrap.cc:492] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc: No bootstrap required, opened a new log
I20260806 18:34:41.924759 21389 ts_tablet_manager.cc:1397] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:34:41.925196 21389 raft_consensus.cc:348] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.925340 21389 raft_consensus.cc:374] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:41.925397 21389 raft_consensus.cc:729] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4bbef384f34b9ea41ce256fe9c08cc, State: Initialized, Role: FOLLOWER
I20260806 18:34:41.925537 21389 consensus_queue.cc:260] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.925835 21389 ts_tablet_manager.cc:1428] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:34:41.976855 21342 raft_consensus.cc:482] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:41.977015 21342 raft_consensus.cc:504] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.977478 21342 leader_election.cc:288] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807), 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481)
I20260806 18:34:41.977981 21216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d14a212704246fa8069b1a573db7272" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" is_pre_election: true
I20260806 18:34:41.977984 20923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d14a212704246fa8069b1a573db7272" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770f7edf17934c03a6f0fcf28c7943ec" is_pre_election: true
I20260806 18:34:41.978148 20923 raft_consensus.cc:2454] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 0.
I20260806 18:34:41.978148 21216 raft_consensus.cc:2454] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 0.
I20260806 18:34:41.978472 20997 leader_election.cc:302] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e4bbef384f34b9ea41ce256fe9c08cc, c3e7949fb6e94c088283c87b4e716c25; no voters:
I20260806 18:34:41.978684 21342 raft_consensus.cc:2781] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:41.978768 21342 raft_consensus.cc:482] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:41.978801 21342 raft_consensus.cc:3037] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:41.979992 21342 raft_consensus.cc:504] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.980608 21342 leader_election.cc:288] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 election: Requested vote from peers 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807), 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481)
I20260806 18:34:41.980835 20923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d14a212704246fa8069b1a573db7272" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770f7edf17934c03a6f0fcf28c7943ec"
I20260806 18:34:41.980908 21216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d14a212704246fa8069b1a573db7272" candidate_uuid: "c3e7949fb6e94c088283c87b4e716c25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc"
I20260806 18:34:41.980995 20923 raft_consensus.cc:3037] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:41.981068 21216 raft_consensus.cc:3037] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:41.982121 21216 raft_consensus.cc:2454] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 1.
I20260806 18:34:41.982121 20923 raft_consensus.cc:2454] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e7949fb6e94c088283c87b4e716c25 in term 1.
I20260806 18:34:41.982479 20997 leader_election.cc:302] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e4bbef384f34b9ea41ce256fe9c08cc, c3e7949fb6e94c088283c87b4e716c25; no voters:
I20260806 18:34:41.982699 21342 raft_consensus.cc:2781] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:41.982790 21342 raft_consensus.cc:686] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 1 LEADER]: Becoming Leader. State: Replica: c3e7949fb6e94c088283c87b4e716c25, State: Running, Role: LEADER
I20260806 18:34:41.982949 21342 consensus_queue.cc:237] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } }
I20260806 18:34:41.984696 20757 catalog_manager.cc:5495] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3e7949fb6e94c088283c87b4e716c25 (127.18.125.130). New cstate: current_term: 1 leader_uuid: "c3e7949fb6e94c088283c87b4e716c25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3e7949fb6e94c088283c87b4e716c25" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:42.013139 21216 raft_consensus.cc:1261] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Refusing update from remote peer c3e7949fb6e94c088283c87b4e716c25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:42.013167 20923 raft_consensus.cc:1261] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Refusing update from remote peer c3e7949fb6e94c088283c87b4e716c25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:42.013643 21371 consensus_queue.cc:1035] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e4bbef384f34b9ea41ce256fe9c08cc" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:42.013835 21342 consensus_queue.cc:1035] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "770f7edf17934c03a6f0fcf28c7943ec" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 32807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:34:42.028025 21358 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 18:34:43.586006 20757 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:43.586174 20757 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:43.588862 20757 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb5bb9c1be9f4c1ba85db1b18365a822: Sending DeleteTablet for 3 replicas of tablet 5d14a212704246fa8069b1a573db7272
I20260806 18:34:43.589445 20905 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d14a212704246fa8069b1a573db7272 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:43 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:51928
I20260806 18:34:43.589583 21049 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d14a212704246fa8069b1a573db7272 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:43 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:36746
I20260806 18:34:43.589591 21200 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d14a212704246fa8069b1a573db7272 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:43 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:59008
I20260806 18:34:43.589886 21431 tablet_replica.cc:330] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec: stopping tablet replica
I20260806 18:34:43.589991 21432 tablet_replica.cc:330] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25: stopping tablet replica
I20260806 18:34:43.590044 21431 raft_consensus.cc:2229] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:43.590116 21432 raft_consensus.cc:2229] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:43.590175 21431 raft_consensus.cc:2258] T 5d14a212704246fa8069b1a573db7272 P 770f7edf17934c03a6f0fcf28c7943ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:43.590296 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 20835
I20260806 18:34:43.590327 21432 raft_consensus.cc:2258] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:43.592859 21433 tablet_replica.cc:330] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc: stopping tablet replica
I20260806 18:34:43.593401 21432 ts_tablet_manager.cc:1905] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:43.595808 21432 ts_tablet_manager.cc:1918] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:34:43.595908 21432 log.cc:1196] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/5d14a212704246fa8069b1a573db7272
I20260806 18:34:43.596295 21432 ts_tablet_manager.cc:1939] T 5d14a212704246fa8069b1a573db7272 P c3e7949fb6e94c088283c87b4e716c25: Deleting consensus metadata
I20260806 18:34:43.597048 20742 catalog_manager.cc:4841] TS c3e7949fb6e94c088283c87b4e716c25 (127.18.125.130:38845): tablet 5d14a212704246fa8069b1a573db7272 (table test-table [id=69b0205065174a8487dc2bb34fbbdabc]) successfully deleted
I20260806 18:34:43.597126 21433 raft_consensus.cc:2229] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:43.597312 21433 raft_consensus.cc:2258] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:43.597929 21433 ts_tablet_manager.cc:1905] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:43.599995 21433 ts_tablet_manager.cc:1918] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:34:43.600085 21433 log.cc:1196] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/5d14a212704246fa8069b1a573db7272
I20260806 18:34:43.600339 21433 ts_tablet_manager.cc:1939] T 5d14a212704246fa8069b1a573db7272 P 2e4bbef384f34b9ea41ce256fe9c08cc: Deleting consensus metadata
I20260806 18:34:43.601249 20742 catalog_manager.cc:4841] TS 2e4bbef384f34b9ea41ce256fe9c08cc (127.18.125.131:40481): tablet 5d14a212704246fa8069b1a573db7272 (table test-table [id=69b0205065174a8487dc2bb34fbbdabc]) successfully deleted
W20260806 18:34:43.602468 20743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:32807 (error 108)
I20260806 18:34:43.602674 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 20980
I20260806 18:34:43.609076 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:43.608670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:32807 (local address 127.0.0.1:60868)
0806 18:34:43.608917 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:43.608988 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:32807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":100,"threads_started":1}
W20260806 18:34:43.609213 20743 catalog_manager.cc:4584] TS 770f7edf17934c03a6f0fcf28c7943ec (127.18.125.129:32807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5d14a212704246fa8069b1a573db7272: Network error: Client connection negotiation failed: client connection to 127.18.125.129:32807: connect: Connection refused (error 111)
I20260806 18:34:43.611642 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 21129
I20260806 18:34:43.620390 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 20723
2026-08-06T18:34:43Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6505 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:43 dist-test-slave-8w9v krb5kdc[21438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:43 dist-test-slave-8w9v krb5kdc[21438](info): set up 2 sockets
Aug 06 18:34:43 dist-test-slave-8w9v krb5kdc[21438](info): commencing operation
krb5kdc: starting...
W20260806 18:34:44.702486 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:34:44 dist-test-slave-8w9v krb5kdc[21438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:34:44.841501 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:35037
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:35037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:44.964637 21451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:44.964952 21451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:44.964996 21451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:44.969239 21451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:44.969303 21451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:44.969323 21451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:44.969341 21451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:44.969357 21451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:44.974819 21451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:35037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:35037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:44.975919 21451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:44.976998 21451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:44.986184 21456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:44.986223 21451 server_base.cc:1034] running on GCE node
W20260806 18:34:44.986246 21457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:44.986230 21459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:44.987069 21451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:44.988309 21451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:44.989476 21451 hybrid_clock.cc:648] HybridClock initialized: now 1786041284989464 us; error 26 us; skew 500 ppm
Aug 06 18:34:44 dist-test-slave-8w9v krb5kdc[21438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041284, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:44.992460 21451 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:34:44.993387 21451 webserver.cc:460] Webserver started at http://127.18.125.190:40303/ using document root <none> and password file <none>
I20260806 18:34:44.993983 21451 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:44.994048 21451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:44.994282 21451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:44.996532 21451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "f4cd7cfa5c044d98b438a1aabee38673"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-8w9v"
server_key: "173107324c8c81069937cea71cd08342"
server_key_iv: "e8a40a5c29f445a2024468d01b521a37"
server_key_version: "encryptionkey@0"
I20260806 18:34:44.997121 21451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "f4cd7cfa5c044d98b438a1aabee38673"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-8w9v"
server_key: "173107324c8c81069937cea71cd08342"
server_key_iv: "e8a40a5c29f445a2024468d01b521a37"
server_key_version: "encryptionkey@0"
I20260806 18:34:45.001044 21451 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:34:45.003530 21466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:45.004909 21451 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:34:45.005044 21451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "f4cd7cfa5c044d98b438a1aabee38673"
format_stamp: "Formatted at 2026-08-06 18:34:44 on dist-test-slave-8w9v"
server_key: "173107324c8c81069937cea71cd08342"
server_key_iv: "e8a40a5c29f445a2024468d01b521a37"
server_key_version: "encryptionkey@0"
I20260806 18:34:45.005154 21451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:45.029595 21451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:45.033598 21451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:45.033849 21451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:45.041658 21451 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:35037
I20260806 18:34:45.041682 21520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:35037 every 8 connection(s)
I20260806 18:34:45.042665 21451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:45.045485 21521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:45.049103 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 21451
I20260806 18:34:45.049197 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:45.049475 18934 external_mini_cluster.cc:1428] Setting key 3d1b2d1866a6ab2cb31de48d36faa968
I20260806 18:34:45.051743 21521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673: Bootstrap starting.
I20260806 18:34:45.054669 21521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:45.055588 21521 log.cc:824] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:45.058089 21521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673: No bootstrap required, opened a new log
Aug 06 18:34:45 dist-test-slave-8w9v krb5kdc[21438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:45.062354 21521 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd7cfa5c044d98b438a1aabee38673" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35037 } }
I20260806 18:34:45.062660 21521 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:45.062723 21521 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4cd7cfa5c044d98b438a1aabee38673, State: Initialized, Role: FOLLOWER
I20260806 18:34:45.063217 21521 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd7cfa5c044d98b438a1aabee38673" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35037 } }
I20260806 18:34:45.063346 21521 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:45.063421 21521 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:45.063536 21521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:45.064570 21521 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd7cfa5c044d98b438a1aabee38673" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35037 } }
I20260806 18:34:45.064949 21521 leader_election.cc:302] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4cd7cfa5c044d98b438a1aabee38673; no voters:
I20260806 18:34:45.065523 21521 leader_election.cc:288] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:45.065622 21527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:45.065809 21527 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [term 1 LEADER]: Becoming Leader. State: Replica: f4cd7cfa5c044d98b438a1aabee38673, State: Running, Role: LEADER
I20260806 18:34:45.066206 21527 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd7cfa5c044d98b438a1aabee38673" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35037 } }
I20260806 18:34:45.066565 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.050979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50592 (local address 127.18.125.190:35037)
0806 18:34:45.051432 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.051452 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.051495 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:34:45.052365 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.052391 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.052811 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.053214 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:45.053918 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.055452 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.056629 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.057301 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.057467 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.060248 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:45.060255 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:45.060272 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:45.060304 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:45.062907 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.063513 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.063519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.063526 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.063689 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.063995 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.064000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.064001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.064505 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:45.064731 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.065130 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.065472 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":125,"threads_started":1}
I20260806 18:34:45.066947 21521 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:45.067556 21527 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4cd7cfa5c044d98b438a1aabee38673. Latest consensus state: current_term: 1 leader_uuid: "f4cd7cfa5c044d98b438a1aabee38673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd7cfa5c044d98b438a1aabee38673" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35037 } } }
I20260806 18:34:45.068001 21528 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4cd7cfa5c044d98b438a1aabee38673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4cd7cfa5c044d98b438a1aabee38673" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35037 } } }
I20260806 18:34:45.069053 21528 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:45.069342 21527 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673 [sys.catalog]: This master's current role is: LEADER
W20260806 18:34:45.071842 21542 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:34:45.071941 21542 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:34:45.072031 21543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:45.072799 21543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:45.078473 21543 catalog_manager.cc:1347] Generated new cluster ID: e76479ed72a248aab57f07372eb52389
I20260806 18:34:45.078547 21543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:45.098841 21543 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:45.100246 21543 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:45.107275 21543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4cd7cfa5c044d98b438a1aabee38673: Generated new TSK 0
I20260806 18:34:45.108107 21543 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:34:45.140173 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35037
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:45.258633 21552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:45.259012 21552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:45.259105 21552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:45.263427 21552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:45.263545 21552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:45.263648 21552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:34:45.269763 21552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:45.271013 21552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:45.272104 21552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:45.282182 21560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:45.282212 21558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:45.282269 21557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:45.282325 21552 server_base.cc:1034] running on GCE node
I20260806 18:34:45.282932 21552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:45.283592 21552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:45.284787 21552 hybrid_clock.cc:648] HybridClock initialized: now 1786041285284774 us; error 25 us; skew 500 ppm
Aug 06 18:34:45 dist-test-slave-8w9v krb5kdc[21438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:45.287710 21552 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:34:45.288641 21552 webserver.cc:460] Webserver started at http://127.18.125.129:34655/ using document root <none> and password file <none>
I20260806 18:34:45.289220 21552 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:45.289309 21552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:45.289544 21552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:45.291618 21552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "b77c41e5b5914cabb08b96ec91755e57"
format_stamp: "Formatted at 2026-08-06 18:34:45 on dist-test-slave-8w9v"
server_key: "5032c6ec4e2db42924e02f532de28f51"
server_key_iv: "b7220e5518758b5363e3d5385c46df88"
server_key_version: "encryptionkey@0"
I20260806 18:34:45.292259 21552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "b77c41e5b5914cabb08b96ec91755e57"
format_stamp: "Formatted at 2026-08-06 18:34:45 on dist-test-slave-8w9v"
server_key: "5032c6ec4e2db42924e02f532de28f51"
server_key_iv: "b7220e5518758b5363e3d5385c46df88"
server_key_version: "encryptionkey@0"
I20260806 18:34:45.296028 21552 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:45.298556 21568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:45.299760 21552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:34:45.299916 21552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "b77c41e5b5914cabb08b96ec91755e57"
format_stamp: "Formatted at 2026-08-06 18:34:45 on dist-test-slave-8w9v"
server_key: "5032c6ec4e2db42924e02f532de28f51"
server_key_iv: "b7220e5518758b5363e3d5385c46df88"
server_key_version: "encryptionkey@0"
I20260806 18:34:45.300036 21552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:45.324497 21552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:45.327898 21552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:45.328150 21552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:45.328789 21552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:45.329731 21552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:45.329803 21552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:45.329887 21552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:45.329926 21552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:45.339855 21552 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:34437
I20260806 18:34:45.339880 21689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:34437 every 8 connection(s)
I20260806 18:34:45.340934 21552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:45.348112 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 21552
I20260806 18:34:45.348340 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:45.348753 18934 external_mini_cluster.cc:1428] Setting key 7a18ecc664079e030eca057907c8a57b
Aug 06 18:34:45 dist-test-slave-8w9v krb5kdc[21438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:45.357511 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.342912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:43585 (local address 127.18.125.190:35037)
0806 18:34:45.343080 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.343085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.344326 (+ 1241us) server_negotiation.cc:407] Connection header received
0806 18:34:45.345285 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.345288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.345339 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.345425 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:45.346650 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.347211 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.348538 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.348987 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.349147 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.352602 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:45.352611 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:45.352617 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:45.352661 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:45.354432 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.355309 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.355312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.355313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.355402 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.355917 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.355919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.355920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.356284 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:45.356372 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.357239 (+ 867us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.357371 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:34:45.359143 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.343363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:35037 (local address 127.18.125.129:43585)
0806 18:34:45.344084 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.344197 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:45.345049 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:45.345594 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:45.345601 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:45.345981 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:45.346484 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.346498 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.347368 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.347371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.348369 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.348385 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.349188 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.349191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.349392 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.350141 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:45.350164 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:45.352343 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:45.354591 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:45.354603 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:45.354629 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:45.355164 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:45.355510 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:45.355516 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:45.355522 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:45.355801 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:45.356418 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:45.356429 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:45.356878 (+ 449us) client_negotiation.cc:769] Sending connection context
0806 18:34:45.357193 (+ 315us) client_negotiation.cc:240] Negotiation successful
0806 18:34:45.357722 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":126,"threads_started":1}
I20260806 18:34:45.360759 21690 heartbeater.cc:347] Connected to a master server at 127.18.125.190:35037
I20260806 18:34:45.361056 21690 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:34:45.361640 21690 heartbeater.cc:511] Master 127.18.125.190:35037 requested a full tablet report, sending...
I20260806 18:34:45.363365 21485 ts_manager.cc:194] Registered new tserver with Master: b77c41e5b5914cabb08b96ec91755e57 (127.18.125.129:34437)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
I20260806 18:34:45.364730 21485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:43585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:34:45.383204 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35037
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:45.516711 21695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:45.517264 21695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:45.517405 21695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:45.525354 21695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:45.525463 21695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:45.525573 21695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:34:45.536468 21695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:45.538028 21695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:45.539224 21695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:45.551906 21702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:45.552116 21695 server_base.cc:1034] running on GCE node
W20260806 18:34:45.551909 21703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:45.552349 21705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:45.552934 21695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:45.553735 21695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:45.554939 21695 hybrid_clock.cc:648] HybridClock initialized: now 1786041285554904 us; error 45 us; skew 500 ppm
Aug 06 18:34:45 dist-test-slave-8w9v krb5kdc[21438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:45.558641 21695 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:34:45.559854 21695 webserver.cc:460] Webserver started at http://127.18.125.130:35215/ using document root <none> and password file <none>
I20260806 18:34:45.560793 21695 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:45.560905 21695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:45.561146 21695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:45.563613 21695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "16afd85f9bc94323b44653fe8a6dbdbf"
format_stamp: "Formatted at 2026-08-06 18:34:45 on dist-test-slave-8w9v"
server_key: "f4fdf899b064720ed81dcaccb79f7065"
server_key_iv: "5d4fd414aee90f923c2fb194540ae044"
server_key_version: "encryptionkey@0"
I20260806 18:34:45.564281 21695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "16afd85f9bc94323b44653fe8a6dbdbf"
format_stamp: "Formatted at 2026-08-06 18:34:45 on dist-test-slave-8w9v"
server_key: "f4fdf899b064720ed81dcaccb79f7065"
server_key_iv: "5d4fd414aee90f923c2fb194540ae044"
server_key_version: "encryptionkey@0"
I20260806 18:34:45.569442 21695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 18:34:45.572058 21712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:45.573494 21695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:45.573642 21695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "16afd85f9bc94323b44653fe8a6dbdbf"
format_stamp: "Formatted at 2026-08-06 18:34:45 on dist-test-slave-8w9v"
server_key: "f4fdf899b064720ed81dcaccb79f7065"
server_key_iv: "5d4fd414aee90f923c2fb194540ae044"
server_key_version: "encryptionkey@0"
I20260806 18:34:45.573761 21695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:45.588007 21695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:45.591492 21695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:45.591715 21695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:45.592468 21695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:45.593386 21695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:45.593456 21695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:45.593529 21695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:45.593577 21695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:45.604563 21695 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:41733
I20260806 18:34:45.604626 21826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:41733 every 8 connection(s)
I20260806 18:34:45.605551 21695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:45.614300 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 21695
I20260806 18:34:45.614431 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:45.614768 18934 external_mini_cluster.cc:1428] Setting key ded7d2b39a4e5824f237e0e69db55a4f
Aug 06 18:34:45 dist-test-slave-8w9v krb5kdc[21438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:45.625244 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.607883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:46969 (local address 127.18.125.190:35037)
0806 18:34:45.608046 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.608051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.609653 (+ 1602us) server_negotiation.cc:407] Connection header received
0806 18:34:45.610943 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.610946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.611014 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.611116 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:45.613651 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.614545 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.616116 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.616376 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.616511 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.619815 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:45.619823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:45.619829 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:45.619867 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:45.622312 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.623063 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.623068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.623070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.623171 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.623632 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.623636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.623638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.623894 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:45.623973 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.624818 (+ 845us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.624983 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:34:45.626425 21830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.608414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:35037 (local address 127.18.125.130:46969)
0806 18:34:45.609439 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.609509 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:45.610671 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:45.611779 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:45.611796 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:45.612357 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:45.613413 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.613443 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.614737 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.614741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.615973 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.615986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.616596 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.616601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.616857 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.617732 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:45.617760 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:45.619596 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:45.622492 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:45.622499 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:45.622520 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:45.622926 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:45.623294 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:45.623298 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:45.623300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:45.623502 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:45.624072 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:45.624079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:45.624492 (+ 413us) client_negotiation.cc:769] Sending connection context
0806 18:34:45.624728 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 18:34:45.625122 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":108,"threads_started":1}
I20260806 18:34:45.627840 21827 heartbeater.cc:347] Connected to a master server at 127.18.125.190:35037
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:34:45.628110 21827 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:45.628788 21827 heartbeater.cc:511] Master 127.18.125.190:35037 requested a full tablet report, sending...
I20260806 18:34:45.630088 21485 ts_manager.cc:194] Registered new tserver with Master: 16afd85f9bc94323b44653fe8a6dbdbf (127.18.125.130:41733)
I20260806 18:34:45.630791 21485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:46969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:34:45.651620 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35037
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:45.780028 21835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:45.780400 21835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:45.780475 21835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:45.785023 21835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:45.785099 21835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:45.785192 21835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:34:45.791181 21835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:45.792407 21835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:45.793484 21835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:45.804627 21841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:45.804740 21843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:45.804919 21840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:45.805816 21835 server_base.cc:1034] running on GCE node
I20260806 18:34:45.806367 21835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:45.807113 21835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:45.808320 21835 hybrid_clock.cc:648] HybridClock initialized: now 1786041285808261 us; error 59 us; skew 500 ppm
Aug 06 18:34:45 dist-test-slave-8w9v krb5kdc[21438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:45.812719 21835 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:34:45.814716 21835 webserver.cc:460] Webserver started at http://127.18.125.131:45411/ using document root <none> and password file <none>
I20260806 18:34:45.815630 21835 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:45.815716 21835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:45.816031 21835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:45.819114 21835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "c49e0fed30b448b6b0af8aac12bb0641"
format_stamp: "Formatted at 2026-08-06 18:34:45 on dist-test-slave-8w9v"
server_key: "5a4849327d37d4ba5bc16c001a1ea67b"
server_key_iv: "cb97465bbfa921b36774751c39f2442b"
server_key_version: "encryptionkey@0"
I20260806 18:34:45.819891 21835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "c49e0fed30b448b6b0af8aac12bb0641"
format_stamp: "Formatted at 2026-08-06 18:34:45 on dist-test-slave-8w9v"
server_key: "5a4849327d37d4ba5bc16c001a1ea67b"
server_key_iv: "cb97465bbfa921b36774751c39f2442b"
server_key_version: "encryptionkey@0"
I20260806 18:34:45.824810 21835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 18:34:45.827881 21850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:45.829341 21835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:45.829491 21835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "c49e0fed30b448b6b0af8aac12bb0641"
format_stamp: "Formatted at 2026-08-06 18:34:45 on dist-test-slave-8w9v"
server_key: "5a4849327d37d4ba5bc16c001a1ea67b"
server_key_iv: "cb97465bbfa921b36774751c39f2442b"
server_key_version: "encryptionkey@0"
I20260806 18:34:45.829661 21835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:45.854331 21835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:45.858902 21835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:45.859155 21835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:45.859894 21835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:45.861202 21835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:45.861289 21835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:45.861368 21835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:45.861402 21835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:45.873805 21835 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:38233
I20260806 18:34:45.873881 21967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:38233 every 8 connection(s)
I20260806 18:34:45.874878 21835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:45.880668 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 21835
I20260806 18:34:45.880836 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:45.881186 18934 external_mini_cluster.cc:1428] Setting key 70626318571dfe9071eb462a30348c51
Aug 06 18:34:45 dist-test-slave-8w9v krb5kdc[21438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:45.891800 21524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.877237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:52109 (local address 127.18.125.190:35037)
0806 18:34:45.877419 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.877426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.878681 (+ 1255us) server_negotiation.cc:407] Connection header received
0806 18:34:45.879645 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.879650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.879732 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.879852 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:45.881200 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.881969 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.883772 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.884000 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.884084 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.887305 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:45.887311 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:45.887314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:45.887351 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:45.889231 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.889855 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.889860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.889862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.889979 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:45.890384 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:45.890387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:45.890389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:45.890670 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:45.890810 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:45.891428 (+ 618us) server_negotiation.cc:299] Negotiation successful
0806 18:34:45.891585 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:34:45.892705 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:45.877689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:35037 (local address 127.18.125.131:52109)
0806 18:34:45.878497 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:45.878552 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:45.879426 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:45.880027 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:45.880034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:45.880473 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:45.881045 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.881061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.882122 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.882126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.883609 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:45.883624 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:45.884193 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:45.884198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:45.884444 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:45.885179 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:45.885207 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:45.887091 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:45.889392 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:45.889399 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:45.889414 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:45.889740 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:45.890121 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:45.890124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:45.890125 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:45.890275 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:45.890860 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:45.890866 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:45.891152 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 18:34:45.891339 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 18:34:45.891667 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":159,"threads_started":1}
I20260806 18:34:45.894060 21968 heartbeater.cc:347] Connected to a master server at 127.18.125.190:35037
I20260806 18:34:45.894356 21968 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:45.895083 21968 heartbeater.cc:511] Master 127.18.125.190:35037 requested a full tablet report, sending...
I20260806 18:34:45.896378 21485 ts_manager.cc:194] Registered new tserver with Master: c49e0fed30b448b6b0af8aac12bb0641 (127.18.125.131:38233)
I20260806 18:34:45.896984 21485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:52109
I20260806 18:34:45.907480 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:34:45.923947 21524 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:45.922175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50596 (local address 127.18.125.190:35037)
0806 18:34:45.922338 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:45.922344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:45.922507 (+ 163us) server_negotiation.cc:407] Connection header received
0806 18:34:45.922624 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:45.922627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:45.922684 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:45.922800 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:45.923786 (+ 986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50596: BlockingRecv error: recv got EOF from 127.0.0.1:50596 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:34:45.925776 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 21552
I20260806 18:34:45.933233 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 21695
I20260806 18:34:45.941159 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 21835
I20260806 18:34:45.948030 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 21451
2026-08-06T18:34:45Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2325 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:46 dist-test-slave-8w9v krb5kdc[21980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:46 dist-test-slave-8w9v krb5kdc[21980](info): set up 2 sockets
Aug 06 18:34:46 dist-test-slave-8w9v krb5kdc[21980](info): commencing operation
krb5kdc: starting...
W20260806 18:34:47.059147 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:34:47 dist-test-slave-8w9v krb5kdc[21980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:34:47.195854 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34179
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:46823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:34179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:47.316605 21994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:47.316952 21994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:47.317039 21994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:47.321264 21994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:47.321374 21994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:47.321412 21994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:47.321461 21994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:47.321514 21994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:47.327976 21994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:46823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:34179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:47.329375 21994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:47.330508 21994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:47.339717 22000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:47.339708 22002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:47.339708 21999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:47.340157 21994 server_base.cc:1034] running on GCE node
I20260806 18:34:47.340741 21994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:47.341990 21994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:47.343169 21994 hybrid_clock.cc:648] HybridClock initialized: now 1786041287343132 us; error 64 us; skew 500 ppm
Aug 06 18:34:47 dist-test-slave-8w9v krb5kdc[21980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041287, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:47.350768 21994 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:34:47.351854 21994 webserver.cc:460] Webserver started at http://127.18.125.190:36283/ using document root <none> and password file <none>
I20260806 18:34:47.352694 21994 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:47.352799 21994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:47.353047 21994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:47.355247 21994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "8e59eb9724684067bef4c64fb7145182"
format_stamp: "Formatted at 2026-08-06 18:34:47 on dist-test-slave-8w9v"
server_key: "ff0e48fe3ba63242ffba00586ce7f9ed"
server_key_iv: "8362dc4f2271e8a3e180e7a8986446a7"
server_key_version: "encryptionkey@0"
I20260806 18:34:47.355840 21994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "8e59eb9724684067bef4c64fb7145182"
format_stamp: "Formatted at 2026-08-06 18:34:47 on dist-test-slave-8w9v"
server_key: "ff0e48fe3ba63242ffba00586ce7f9ed"
server_key_iv: "8362dc4f2271e8a3e180e7a8986446a7"
server_key_version: "encryptionkey@0"
I20260806 18:34:47.359820 21994 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:34:47.362726 22009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:47.364037 21994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:47.364213 21994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "8e59eb9724684067bef4c64fb7145182"
format_stamp: "Formatted at 2026-08-06 18:34:47 on dist-test-slave-8w9v"
server_key: "ff0e48fe3ba63242ffba00586ce7f9ed"
server_key_iv: "8362dc4f2271e8a3e180e7a8986446a7"
server_key_version: "encryptionkey@0"
I20260806 18:34:47.364346 21994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:47.379024 21994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:47.382369 21994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:47.382598 21994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:47.390240 21994 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:34179
I20260806 18:34:47.390566 22063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:34179 every 8 connection(s)
I20260806 18:34:47.391319 21994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:47.393250 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 21994
I20260806 18:34:47.393375 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:47.393687 18934 external_mini_cluster.cc:1428] Setting key d52462d4118c1868d5902a7246cdd3c7
I20260806 18:34:47.394878 22064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:47.400943 22064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182: Bootstrap starting.
Aug 06 18:34:47 dist-test-slave-8w9v krb5kdc[21980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:47.403635 22064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:47.404477 22064 log.cc:824] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:47.406985 22064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182: No bootstrap required, opened a new log
I20260806 18:34:47.408914 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:47.395201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57668 (local address 127.18.125.190:34179)
0806 18:34:47.395669 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:47.395684 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:47.395719 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:34:47.396455 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:47.396473 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:47.396809 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:47.397156 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:47.398003 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:47.398993 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:47.399861 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:47.400406 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:47.400607 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:47.403492 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:47.403497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:47.403511 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:47.403544 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:47.405604 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:47.406096 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:47.406103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:47.406109 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:47.406272 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:47.406551 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:47.406555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:47.406556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:47.407113 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:47.407300 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:47.407578 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 18:34:47.407913 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":158,"threads_started":1}
I20260806 18:34:47.411769 22064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e59eb9724684067bef4c64fb7145182" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34179 } }
I20260806 18:34:47.412086 22064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:47.412099 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:47.412194 22064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e59eb9724684067bef4c64fb7145182, State: Initialized, Role: FOLLOWER
I20260806 18:34:47.412635 22064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e59eb9724684067bef4c64fb7145182" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34179 } }
I20260806 18:34:47.412787 22064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:47.412873 22064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:47.412978 22064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:47.413976 22064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e59eb9724684067bef4c64fb7145182" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34179 } }
I20260806 18:34:47.414347 22064 leader_election.cc:302] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e59eb9724684067bef4c64fb7145182; no voters:
I20260806 18:34:47.414697 22064 leader_election.cc:288] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:47.414814 22071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:47.415053 22071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [term 1 LEADER]: Becoming Leader. State: Replica: 8e59eb9724684067bef4c64fb7145182, State: Running, Role: LEADER
I20260806 18:34:47.415421 22071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e59eb9724684067bef4c64fb7145182" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34179 } }
I20260806 18:34:47.416178 22064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:47.416786 22073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e59eb9724684067bef4c64fb7145182. Latest consensus state: current_term: 1 leader_uuid: "8e59eb9724684067bef4c64fb7145182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e59eb9724684067bef4c64fb7145182" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34179 } } }
I20260806 18:34:47.416911 22073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:47.417202 22072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e59eb9724684067bef4c64fb7145182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e59eb9724684067bef4c64fb7145182" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34179 } } }
I20260806 18:34:47.417335 22072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:47.417546 22076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:47.421167 22076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:47.427214 22076 catalog_manager.cc:1347] Generated new cluster ID: c803bb271a294c8fad2e6e3e050fa5d9
I20260806 18:34:47.427457 22076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:47.433816 22067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:47.428863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57678 (local address 127.18.125.190:34179)
0806 18:34:47.429014 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:47.429019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:47.429264 (+ 245us) server_negotiation.cc:407] Connection header received
0806 18:34:47.429438 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:47.429441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:47.429495 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:47.429580 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:47.430117 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:47.430671 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:47.431533 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:47.431799 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:47.431917 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:47.432843 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:47.432847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:47.432850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:47.433342 (+ 492us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:47.433373 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:47.433390 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:47.433498 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:47.433582 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 18:34:47.433647 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:34:47.452231 22076 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:47.453346 22076 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:47.466643 22076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e59eb9724684067bef4c64fb7145182: Generated new TSK 0
I20260806 18:34:47.467636 22076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:47.477435 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 21994
2026-08-06T18:34:47Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1527 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:47 dist-test-slave-8w9v krb5kdc[22099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:47 dist-test-slave-8w9v krb5kdc[22099](info): set up 2 sockets
Aug 06 18:34:47 dist-test-slave-8w9v krb5kdc[22099](info): commencing operation
krb5kdc: starting...
W20260806 18:34:48.566401 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:34:48 dist-test-slave-8w9v krb5kdc[22099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:34:48.704873 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:40743
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:40743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:48.827783 22113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:48.828099 22113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:48.828231 22113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:48.832707 22113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:48.832795 22113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:48.832816 22113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:48.832834 22113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:48.832850 22113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:48.838618 22113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:40743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:40743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:48.839751 22113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:48.840849 22113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:48.850415 22121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:48.850416 22118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:48.850586 22113 server_base.cc:1034] running on GCE node
W20260806 18:34:48.850416 22119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:48.851370 22113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:48.852708 22113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:48.853888 22113 hybrid_clock.cc:648] HybridClock initialized: now 1786041288853865 us; error 42 us; skew 500 ppm
Aug 06 18:34:48 dist-test-slave-8w9v krb5kdc[22099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041288, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:48.856951 22113 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:34:48.857960 22113 webserver.cc:460] Webserver started at http://127.18.125.190:37681/ using document root <none> and password file <none>
I20260806 18:34:48.858567 22113 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:48.858664 22113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:48.858914 22113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:48.861159 22113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "e67d4b4914d344c081d47282a56fe40c"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-8w9v"
server_key: "64506a6f241b3858ab91205581dd26ef"
server_key_iv: "72bcdebbb3479383a7bb06fc8a84ed0b"
server_key_version: "encryptionkey@0"
I20260806 18:34:48.861763 22113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "e67d4b4914d344c081d47282a56fe40c"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-8w9v"
server_key: "64506a6f241b3858ab91205581dd26ef"
server_key_iv: "72bcdebbb3479383a7bb06fc8a84ed0b"
server_key_version: "encryptionkey@0"
I20260806 18:34:48.866099 22113 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:48.868903 22128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:48.870265 22113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:48.870435 22113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "e67d4b4914d344c081d47282a56fe40c"
format_stamp: "Formatted at 2026-08-06 18:34:48 on dist-test-slave-8w9v"
server_key: "64506a6f241b3858ab91205581dd26ef"
server_key_iv: "72bcdebbb3479383a7bb06fc8a84ed0b"
server_key_version: "encryptionkey@0"
I20260806 18:34:48.870579 22113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:48.885855 22113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:48.889218 22113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:48.889446 22113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:48.897956 22113 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:40743
I20260806 18:34:48.898321 22182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:40743 every 8 connection(s)
I20260806 18:34:48.899155 22113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:48.902472 22183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:48.902912 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 22113
I20260806 18:34:48.903065 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:48.903451 18934 external_mini_cluster.cc:1428] Setting key 4e7a40450e31127281bb0a7fabf70cc5
I20260806 18:34:48.911043 22183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c: Bootstrap starting.
Aug 06 18:34:48 dist-test-slave-8w9v krb5kdc[22099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:48.914528 22183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:48.915676 22183 log.cc:824] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:48.918548 22183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c: No bootstrap required, opened a new log
I20260806 18:34:48.921113 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:48.905011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.18.125.190:40743)
0806 18:34:48.905455 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:48.905471 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:48.905508 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:34:48.906631 (+ 1123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:48.906660 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:48.907067 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:48.907371 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:48.908505 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:48.909519 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:48.910803 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:48.911350 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:48.911493 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:48.914628 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:48.914636 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:48.914658 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:48.914702 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:48.916968 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:48.917502 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:48.917509 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:48.917517 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:48.917677 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:48.917978 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:48.917981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:48.917983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:48.918562 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:48.918784 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:48.919571 (+ 787us) server_negotiation.cc:299] Negotiation successful
0806 18:34:48.919939 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":135,"threads_started":1}
I20260806 18:34:48.923763 22183 raft_consensus.cc:348] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d4b4914d344c081d47282a56fe40c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40743 } }
I20260806 18:34:48.924058 22183 raft_consensus.cc:374] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:48.924113 22183 raft_consensus.cc:729] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e67d4b4914d344c081d47282a56fe40c, State: Initialized, Role: FOLLOWER
I20260806 18:34:48.924710 22183 consensus_queue.cc:260] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d4b4914d344c081d47282a56fe40c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40743 } }
I20260806 18:34:48.924891 22183 raft_consensus.cc:388] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:48.924994 22183 raft_consensus.cc:482] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:48.925115 22183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:48.926254 22183 raft_consensus.cc:504] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d4b4914d344c081d47282a56fe40c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40743 } }
I20260806 18:34:48.926669 22183 leader_election.cc:302] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e67d4b4914d344c081d47282a56fe40c; no voters:
I20260806 18:34:48.927050 22183 leader_election.cc:288] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:48.927405 22194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:48.927731 22194 raft_consensus.cc:686] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [term 1 LEADER]: Becoming Leader. State: Replica: e67d4b4914d344c081d47282a56fe40c, State: Running, Role: LEADER
I20260806 18:34:48.928218 22194 consensus_queue.cc:237] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d4b4914d344c081d47282a56fe40c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40743 } }
I20260806 18:34:48.928730 22183 sys_catalog.cc:564] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:48.930235 22195 sys_catalog.cc:455] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e67d4b4914d344c081d47282a56fe40c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d4b4914d344c081d47282a56fe40c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40743 } } }
I20260806 18:34:48.930406 22195 sys_catalog.cc:458] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:48.930850 22203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:48.930755 22196 sys_catalog.cc:455] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e67d4b4914d344c081d47282a56fe40c. Latest consensus state: current_term: 1 leader_uuid: "e67d4b4914d344c081d47282a56fe40c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d4b4914d344c081d47282a56fe40c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40743 } } }
I20260806 18:34:48.931643 22196 sys_catalog.cc:458] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:48.934445 22203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:48.940570 22203 catalog_manager.cc:1347] Generated new cluster ID: 229725fc7a9c4e658c8f6efd72f22c34
I20260806 18:34:48.940665 22203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:48.946738 22203 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:48.948166 22203 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:48.961256 22203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e67d4b4914d344c081d47282a56fe40c: Generated new TSK 0
I20260806 18:34:48.962131 22203 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:34:48.995878 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40743
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:49.122991 22215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:49.123288 22215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:49.123337 22215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:49.127526 22215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:49.127605 22215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:49.127667 22215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:34:49.133636 22215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:49.134800 22215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:49.135857 22215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:49.146726 22222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:49.146829 22221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:49.146675 22224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:49.146951 22215 server_base.cc:1034] running on GCE node
I20260806 18:34:49.147518 22215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:49.148437 22215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:49.149658 22215 hybrid_clock.cc:648] HybridClock initialized: now 1786041289149640 us; error 34 us; skew 500 ppm
Aug 06 18:34:49 dist-test-slave-8w9v krb5kdc[22099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:49.152796 22215 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:34:49.153872 22215 webserver.cc:460] Webserver started at http://127.18.125.129:45589/ using document root <none> and password file <none>
I20260806 18:34:49.154459 22215 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:49.154533 22215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:49.154744 22215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:49.157074 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "e3ed7144e39842148ccbf2e5b97f2003"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-8w9v"
server_key: "a79eaf487324a258846aff4582c0ed83"
server_key_iv: "3c9af2ca3883e7fbc58bb863cf5bc10e"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.157660 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "e3ed7144e39842148ccbf2e5b97f2003"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-8w9v"
server_key: "a79eaf487324a258846aff4582c0ed83"
server_key_iv: "3c9af2ca3883e7fbc58bb863cf5bc10e"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.161916 22215 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:49.164690 22231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.166064 22215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:49.166222 22215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "e3ed7144e39842148ccbf2e5b97f2003"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-8w9v"
server_key: "a79eaf487324a258846aff4582c0ed83"
server_key_iv: "3c9af2ca3883e7fbc58bb863cf5bc10e"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.166357 22215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:49.193050 22215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:49.196815 22215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:49.197048 22215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:49.197732 22215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:49.198660 22215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:49.198709 22215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.198783 22215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:49.198817 22215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.209316 22215 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:36941
I20260806 18:34:49.209347 22349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:36941 every 8 connection(s)
I20260806 18:34:49.210366 22215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:49.214113 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 22215
I20260806 18:34:49.214263 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:49.214619 18934 external_mini_cluster.cc:1428] Setting key 8db48562590e8872ae40d56fa8eac7a9
Aug 06 18:34:49 dist-test-slave-8w9v krb5kdc[22099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:34:49.227274 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.212639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:49965 (local address 127.18.125.190:40743)
0806 18:34:49.212824 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.212830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.213867 (+ 1037us) server_negotiation.cc:407] Connection header received
0806 18:34:49.215762 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.215766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.215823 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.215933 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.217742 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.218374 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.219640 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.219894 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.220035 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.222918 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:49.222921 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:49.222923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:49.222952 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:49.224752 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.225421 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.225425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.225427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.225524 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.225897 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.225900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.225903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.226217 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:49.226337 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:49.226904 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 18:34:49.227052 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 18:34:49.228431 22352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.213114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:40743 (local address 127.18.125.129:49965)
0806 18:34:49.213647 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:49.213697 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:49.215494 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:49.216116 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:49.216175 (+ 59us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:49.216773 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:49.217545 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.217571 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.218536 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.218539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.219482 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.219493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.220094 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.220098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.220341 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.221071 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:49.221090 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:49.222696 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:49.224915 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:49.224924 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:49.224946 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:49.225304 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:49.225627 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:49.225630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:49.225632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:49.225792 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:49.226340 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:49.226350 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:49.226641 (+ 291us) client_negotiation.cc:769] Sending connection context
0806 18:34:49.226858 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 18:34:49.227240 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":157,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
I20260806 18:34:49.229965 22350 heartbeater.cc:347] Connected to a master server at 127.18.125.190:40743
I20260806 18:34:49.230257 22350 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:49.231045 22350 heartbeater.cc:511] Master 127.18.125.190:40743 requested a full tablet report, sending...
I20260806 18:34:49.232965 22145 ts_manager.cc:194] Registered new tserver with Master: e3ed7144e39842148ccbf2e5b97f2003 (127.18.125.129:36941)
I20260806 18:34:49.234164 22145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:49965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:34:49.253518 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40743
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:49.380791 22355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:49.381124 22355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:49.381217 22355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:49.385430 22355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:49.385521 22355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:49.385613 22355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:34:49.391135 22355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:49.392455 22355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:49.393515 22355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:49.403791 22360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:49.403798 22362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:49.403800 22364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:49.403936 22355 server_base.cc:1034] running on GCE node
I20260806 18:34:49.404599 22355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:49.405294 22355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:49.406468 22355 hybrid_clock.cc:648] HybridClock initialized: now 1786041289406435 us; error 43 us; skew 500 ppm
Aug 06 18:34:49 dist-test-slave-8w9v krb5kdc[22099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:49.409703 22355 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:34:49.410730 22355 webserver.cc:460] Webserver started at http://127.18.125.130:41803/ using document root <none> and password file <none>
I20260806 18:34:49.411377 22355 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:49.411449 22355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:49.411689 22355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:49.414076 22355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "7a1ea25c10234c2c8a7feffc52d5cf3d"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-8w9v"
server_key: "65ed85f7f87b06f310069d0523964b55"
server_key_iv: "f4ccba060e48802a3597d7eaed706d3f"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.414727 22355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "7a1ea25c10234c2c8a7feffc52d5cf3d"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-8w9v"
server_key: "65ed85f7f87b06f310069d0523964b55"
server_key_iv: "f4ccba060e48802a3597d7eaed706d3f"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.419039 22355 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 18:34:49.421924 22372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.423337 22355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 18:34:49.423516 22355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "7a1ea25c10234c2c8a7feffc52d5cf3d"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-8w9v"
server_key: "65ed85f7f87b06f310069d0523964b55"
server_key_iv: "f4ccba060e48802a3597d7eaed706d3f"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.423658 22355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:49.462138 22355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:49.465844 22355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:49.466044 22355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:49.466645 22355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:49.467535 22355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:49.467592 22355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.467638 22355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:49.467655 22355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.478128 22355 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:46883
I20260806 18:34:49.478149 22496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:46883 every 8 connection(s)
I20260806 18:34:49.479156 22355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:49.482000 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 22355
I20260806 18:34:49.482134 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:49.482463 18934 external_mini_cluster.cc:1428] Setting key 4fc7afddd2512cd93a2cb72f09bc617f
Aug 06 18:34:49 dist-test-slave-8w9v krb5kdc[22099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:34:49.495168 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.481109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:53961 (local address 127.18.125.190:40743)
0806 18:34:49.481257 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.481261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.482203 (+ 942us) server_negotiation.cc:407] Connection header received
0806 18:34:49.484006 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.484009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.484063 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.484158 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.485367 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.486046 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.487371 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.487792 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.487917 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.491011 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:49.491015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:49.491018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:49.491050 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:49.492880 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.493565 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.493568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.493569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.493651 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.494009 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.494011 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.494012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.494203 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:49.494368 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:49.494923 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 18:34:49.495041 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
I20260806 18:34:49.496439 22500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.481481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:40743 (local address 127.18.125.130:53961)
0806 18:34:49.482020 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:49.482088 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:49.483751 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:49.484349 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:49.484357 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:49.484722 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:49.485212 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.485225 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.486200 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.486203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.487218 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.487229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.487979 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.487983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.488225 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.488927 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:49.488946 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:49.490810 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:49.493034 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:49.493040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:49.493054 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:49.493442 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:49.493751 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:49.493754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:49.493755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:49.493918 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:49.494351 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:49.494361 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:49.494676 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 18:34:49.494869 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 18:34:49.495176 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":118,"threads_started":1}
I20260806 18:34:49.497798 22497 heartbeater.cc:347] Connected to a master server at 127.18.125.190:40743
I20260806 18:34:49.498087 22497 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:49.498755 22497 heartbeater.cc:511] Master 127.18.125.190:40743 requested a full tablet report, sending...
I20260806 18:34:49.499998 22145 ts_manager.cc:194] Registered new tserver with Master: 7a1ea25c10234c2c8a7feffc52d5cf3d (127.18.125.130:46883)
I20260806 18:34:49.500726 22145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:53961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:34:49.516254 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40743
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:49.640183 22503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:49.640516 22503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:49.640620 22503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:49.645210 22503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:49.645296 22503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:49.645413 22503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:34:49.651391 22503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:49.652712 22503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:49.653816 22503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:49.664674 22508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:49.664682 22509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:49.664750 22503 server_base.cc:1034] running on GCE node
W20260806 18:34:49.664682 22513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:49.665442 22503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:49.666149 22503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:49.667346 22503 hybrid_clock.cc:648] HybridClock initialized: now 1786041289667329 us; error 28 us; skew 500 ppm
Aug 06 18:34:49 dist-test-slave-8w9v krb5kdc[22099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:49.670280 22503 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:34:49.671298 22503 webserver.cc:460] Webserver started at http://127.18.125.131:39513/ using document root <none> and password file <none>
I20260806 18:34:49.671931 22503 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:49.672029 22503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:49.672328 22503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:49.674635 22503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "db467f120a1f4a819b66ac543c80369c"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-8w9v"
server_key: "df9f5bc83d56422cfd79d4b87ea4a299"
server_key_iv: "94089a6bbde8a6fffbed05d7ce7ac788"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.675235 22503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "db467f120a1f4a819b66ac543c80369c"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-8w9v"
server_key: "df9f5bc83d56422cfd79d4b87ea4a299"
server_key_iv: "94089a6bbde8a6fffbed05d7ce7ac788"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.679180 22503 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:34:49.681777 22520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.683082 22503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:34:49.683226 22503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "db467f120a1f4a819b66ac543c80369c"
format_stamp: "Formatted at 2026-08-06 18:34:49 on dist-test-slave-8w9v"
server_key: "df9f5bc83d56422cfd79d4b87ea4a299"
server_key_iv: "94089a6bbde8a6fffbed05d7ce7ac788"
server_key_version: "encryptionkey@0"
I20260806 18:34:49.683344 22503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:49.709947 22503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:49.713255 22503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:49.713495 22503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:49.714144 22503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:49.715041 22503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:49.715116 22503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.715190 22503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:49.715233 22503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:49.724659 22503 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:45309
I20260806 18:34:49.724682 22639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:45309 every 8 connection(s)
I20260806 18:34:49.725629 22503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:49.734234 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 22503
I20260806 18:34:49.734381 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:49.734735 18934 external_mini_cluster.cc:1428] Setting key f5b571e2177c6806d753fe92548e88b3
Aug 06 18:34:49 dist-test-slave-8w9v krb5kdc[22099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:49.740959 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.727670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:54643 (local address 127.18.125.190:40743)
0806 18:34:49.727826 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.727831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.728828 (+ 997us) server_negotiation.cc:407] Connection header received
0806 18:34:49.730434 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.730437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.730490 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.730575 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.731908 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.732552 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.733755 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.734016 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.734109 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.737392 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:49.737398 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:49.737404 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:49.737440 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:49.738796 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.739351 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.739354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.739356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.739451 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.739790 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.739793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.739794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.739984 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:49.740066 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:49.740713 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 18:34:49.740834 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:34:49.741855 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.728078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:40743 (local address 127.18.125.131:54643)
0806 18:34:49.728645 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:49.728690 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:49.730185 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:49.730752 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:49.730768 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:49.731246 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:49.731754 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.731769 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.732709 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.732712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.733624 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:49.733635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.734219 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.734225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:49.734501 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.735174 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:49.735204 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:49.737186 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:49.738924 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:49.738930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:49.738945 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:49.739233 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:49.739545 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:49.739548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:49.739549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:49.739693 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:49.740074 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:49.740079 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:49.740459 (+ 380us) client_negotiation.cc:769] Sending connection context
0806 18:34:49.740635 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 18:34:49.740919 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":146,"threads_started":1}
I20260806 18:34:49.743155 22640 heartbeater.cc:347] Connected to a master server at 127.18.125.190:40743
I20260806 18:34:49.743443 22640 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:49.743988 22640 heartbeater.cc:511] Master 127.18.125.190:40743 requested a full tablet report, sending...
I20260806 18:34:49.745221 22145 ts_manager.cc:194] Registered new tserver with Master: db467f120a1f4a819b66ac543c80369c (127.18.125.131:45309)
I20260806 18:34:49.746119 22145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:54643
I20260806 18:34:49.750201 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:34:49 dist-test-slave-8w9v krb5kdc[22099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 18:34:49 dist-test-slave-8w9v krb5kdc[22099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041289, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:49.783456 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:49.771802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42994 (local address 127.18.125.190:40743)
0806 18:34:49.771999 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:49.772003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:49.772026 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:34:49.772209 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:49.772212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:49.772260 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:49.772341 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:49.773147 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.773988 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.775523 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:49.775764 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:49.775867 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:49.779335 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:49.779339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:49.779342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:49.779378 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:49.781192 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.781734 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.781737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.781739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.781858 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:49.782302 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:49.782305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:49.782307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:49.782579 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:49.782700 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:49.783081 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 18:34:49.783268 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 18:34:49.784066 22145 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42994
I20260806 18:34:49.785465 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 22215
I20260806 18:34:49.794751 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 22355
I20260806 18:34:49.805094 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 22503
I20260806 18:34:49.812098 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 22113
2026-08-06T18:34:49Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2338 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:49 dist-test-slave-8w9v krb5kdc[22652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:49 dist-test-slave-8w9v krb5kdc[22652](info): set up 2 sockets
Aug 06 18:34:49 dist-test-slave-8w9v krb5kdc[22652](info): commencing operation
krb5kdc: starting...
W20260806 18:34:50.894874 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:34:50 dist-test-slave-8w9v krb5kdc[22652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:34:51.039216 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:42525
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:42525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:51.157310 22665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:51.157660 22665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:51.157737 22665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:51.162135 22665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:51.162238 22665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:51.162284 22665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:51.162348 22665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:51.162387 22665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:51.168063 22665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:42525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:42525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:51.169354 22665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:51.170431 22665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:51.179652 22671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:51.179647 22674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:51.179642 22670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:51.180038 22665 server_base.cc:1034] running on GCE node
I20260806 18:34:51.180864 22665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:51.182582 22665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:51.183828 22665 hybrid_clock.cc:648] HybridClock initialized: now 1786041291183814 us; error 34 us; skew 500 ppm
Aug 06 18:34:51 dist-test-slave-8w9v krb5kdc[22652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041291, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:51.187347 22665 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:34:51.188446 22665 webserver.cc:460] Webserver started at http://127.18.125.190:45627/ using document root <none> and password file <none>
I20260806 18:34:51.189013 22665 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:51.189072 22665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:51.189249 22665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:51.191339 22665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "88953ee67bf0485eab99ac64e05571b7"
format_stamp: "Formatted at 2026-08-06 18:34:51 on dist-test-slave-8w9v"
server_key: "a0e7a2bef2e13b5e0b4349bfe1516ab2"
server_key_iv: "9912cef19f5b7e60e5baf0b46d0caecd"
server_key_version: "encryptionkey@0"
I20260806 18:34:51.191879 22665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "88953ee67bf0485eab99ac64e05571b7"
format_stamp: "Formatted at 2026-08-06 18:34:51 on dist-test-slave-8w9v"
server_key: "a0e7a2bef2e13b5e0b4349bfe1516ab2"
server_key_iv: "9912cef19f5b7e60e5baf0b46d0caecd"
server_key_version: "encryptionkey@0"
I20260806 18:34:51.195765 22665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:34:51.198978 22682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:51.200838 22665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:51.200990 22665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "88953ee67bf0485eab99ac64e05571b7"
format_stamp: "Formatted at 2026-08-06 18:34:51 on dist-test-slave-8w9v"
server_key: "a0e7a2bef2e13b5e0b4349bfe1516ab2"
server_key_iv: "9912cef19f5b7e60e5baf0b46d0caecd"
server_key_version: "encryptionkey@0"
I20260806 18:34:51.201141 22665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:51.235291 22665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:51.240305 22665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:51.240566 22665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:51.249995 22665 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:42525
I20260806 18:34:51.250023 22745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:42525 every 8 connection(s)
I20260806 18:34:51.251175 22665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:51.254429 22746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:51.257827 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 22665
I20260806 18:34:51.258010 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:51.258549 18934 external_mini_cluster.cc:1428] Setting key 8acd8894d8cb117421696395cb7b4098
I20260806 18:34:51.261662 22746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7: Bootstrap starting.
I20260806 18:34:51.264246 22746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:51.265069 22746 log.cc:824] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:51.267230 22746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7: No bootstrap required, opened a new log
Aug 06 18:34:51 dist-test-slave-8w9v krb5kdc[22652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:51.271953 22746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88953ee67bf0485eab99ac64e05571b7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42525 } }
I20260806 18:34:51.272332 22746 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:51.272439 22746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88953ee67bf0485eab99ac64e05571b7, State: Initialized, Role: FOLLOWER
I20260806 18:34:51.273237 22746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88953ee67bf0485eab99ac64e05571b7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42525 } }
I20260806 18:34:51.273698 22746 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:51.273893 22746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:51.274057 22746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:51.275410 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:51.260871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49084 (local address 127.18.125.190:42525)
0806 18:34:51.261431 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:51.261448 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:51.261489 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:34:51.262274 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:51.262295 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:51.262690 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:51.263061 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:51.263988 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.265182 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.266100 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.266601 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.266787 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:51.269684 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:51.269690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:51.269705 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:51.269737 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:51.271785 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:51.272468 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:51.272476 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:51.272527 (+ 51us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:51.272682 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:51.273016 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:51.273019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:51.273021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:51.273547 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:51.273782 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:51.274188 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 18:34:51.274536 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":167,"threads_started":1}
I20260806 18:34:51.275394 22746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88953ee67bf0485eab99ac64e05571b7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42525 } }
I20260806 18:34:51.275897 22746 leader_election.cc:302] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88953ee67bf0485eab99ac64e05571b7; no voters:
I20260806 18:34:51.276429 22746 leader_election.cc:288] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:51.276543 22752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:51.276762 22752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [term 1 LEADER]: Becoming Leader. State: Replica: 88953ee67bf0485eab99ac64e05571b7, State: Running, Role: LEADER
I20260806 18:34:51.277137 22752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88953ee67bf0485eab99ac64e05571b7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42525 } }
I20260806 18:34:51.277994 22746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:51.279158 22753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88953ee67bf0485eab99ac64e05571b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88953ee67bf0485eab99ac64e05571b7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42525 } } }
I20260806 18:34:51.279314 22753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:51.279666 22754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88953ee67bf0485eab99ac64e05571b7. Latest consensus state: current_term: 1 leader_uuid: "88953ee67bf0485eab99ac64e05571b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88953ee67bf0485eab99ac64e05571b7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42525 } } }
I20260806 18:34:51.279769 22754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:51.279937 22763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:51.283730 22763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:51.290477 22763 catalog_manager.cc:1347] Generated new cluster ID: 4663c8268c1f4dad8a278f750da81f7b
I20260806 18:34:51.290565 22763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:51.302683 22763 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:51.303691 22763 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:51.315274 22763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88953ee67bf0485eab99ac64e05571b7: Generated new TSK 0
I20260806 18:34:51.316109 22763 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:34:51.347905 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42525
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:51.468886 22775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:51.469197 22775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:51.469245 22775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:51.473512 22775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:51.473592 22775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:51.473655 22775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:34:51.479363 22775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:51.480665 22775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:51.481748 22775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:51.491750 22780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:51.491777 22775 server_base.cc:1034] running on GCE node
W20260806 18:34:51.491799 22781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:51.491986 22783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:51.492547 22775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:51.493422 22775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:51.494598 22775 hybrid_clock.cc:648] HybridClock initialized: now 1786041291494565 us; error 44 us; skew 500 ppm
Aug 06 18:34:51 dist-test-slave-8w9v krb5kdc[22652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041291, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:51.497586 22775 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:34:51.498541 22775 webserver.cc:460] Webserver started at http://127.18.125.129:43261/ using document root <none> and password file <none>
I20260806 18:34:51.499142 22775 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:51.499207 22775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:51.499456 22775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:51.501653 22775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "ec0349d2164d4bd081996e0e3cabb572"
format_stamp: "Formatted at 2026-08-06 18:34:51 on dist-test-slave-8w9v"
server_key: "ba3f409bd5df41249d782e6cad9cb78f"
server_key_iv: "707788c02007f42fb9ef7411a45cfb1d"
server_key_version: "encryptionkey@0"
I20260806 18:34:51.502244 22775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "ec0349d2164d4bd081996e0e3cabb572"
format_stamp: "Formatted at 2026-08-06 18:34:51 on dist-test-slave-8w9v"
server_key: "ba3f409bd5df41249d782e6cad9cb78f"
server_key_iv: "707788c02007f42fb9ef7411a45cfb1d"
server_key_version: "encryptionkey@0"
I20260806 18:34:51.506469 22775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:51.509080 22790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:51.510427 22775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:51.510555 22775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "ec0349d2164d4bd081996e0e3cabb572"
format_stamp: "Formatted at 2026-08-06 18:34:51 on dist-test-slave-8w9v"
server_key: "ba3f409bd5df41249d782e6cad9cb78f"
server_key_iv: "707788c02007f42fb9ef7411a45cfb1d"
server_key_version: "encryptionkey@0"
I20260806 18:34:51.510672 22775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:51.528210 22775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:51.531630 22775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:51.531874 22775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:51.532621 22775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:51.533644 22775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:51.533711 22775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:51.533774 22775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:51.533795 22775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:51.543660 22775 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:46097
I20260806 18:34:51.543689 22911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:46097 every 8 connection(s)
I20260806 18:34:51.544763 22775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:51.545980 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 22775
I20260806 18:34:51.546115 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:51.546458 18934 external_mini_cluster.cc:1428] Setting key 90156ab1fff56b0eb752044687b69da5
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
Aug 06 18:34:51 dist-test-slave-8w9v krb5kdc[22652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041291, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
I20260806 18:34:51.561326 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:51.547241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:36781 (local address 127.18.125.190:42525)
0806 18:34:51.547430 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:51.547435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:51.548635 (+ 1200us) server_negotiation.cc:407] Connection header received
0806 18:34:51.550714 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:51.550717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:51.550780 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:51.550879 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:51.552221 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.552847 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.554065 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.554311 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.554431 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:51.557312 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:51.557316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:51.557318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:51.557345 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:51.558897 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:51.559501 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:51.559505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:51.559508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:51.559594 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:51.559986 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:51.559989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:51.559991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:51.560286 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:51.560403 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:51.560994 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 18:34:51.561144 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 18:34:51.562285 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:51.547707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42525 (local address 127.18.125.129:36781)
0806 18:34:51.548441 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:51.548513 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:51.550479 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:51.551089 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:51.551097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:51.551493 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:51.552007 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:51.552021 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.552995 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.552998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:51.553909 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:51.553920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.554488 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.554492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:51.554676 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:51.555417 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:51.555436 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:51.557091 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:51.559046 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:51.559054 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:51.559073 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:51.559378 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:51.559699 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:51.559702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:51.559704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:51.559874 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:51.560407 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:51.560417 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:51.560717 (+ 300us) client_negotiation.cc:769] Sending connection context
0806 18:34:51.560936 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 18:34:51.561320 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":146,"threads_started":1}
I20260806 18:34:51.563896 22912 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42525
I20260806 18:34:51.564227 22912 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:51.564795 22912 heartbeater.cc:511] Master 127.18.125.190:42525 requested a full tablet report, sending...
I20260806 18:34:51.566504 22701 ts_manager.cc:194] Registered new tserver with Master: ec0349d2164d4bd081996e0e3cabb572 (127.18.125.129:46097)
I20260806 18:34:51.567874 22701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:36781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:34:51.581954 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42525
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:51.706246 22917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:51.706547 22917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:51.706593 22917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:51.711012 22917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:51.711086 22917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:51.711150 22917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:34:51.717290 22917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:51.718468 22917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:51.719553 22917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:51.730677 22922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:51.730682 22923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:51.730677 22925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:51.731114 22917 server_base.cc:1034] running on GCE node
I20260806 18:34:51.731639 22917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:51.732486 22917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:51.733721 22917 hybrid_clock.cc:648] HybridClock initialized: now 1786041291733703 us; error 41 us; skew 500 ppm
Aug 06 18:34:51 dist-test-slave-8w9v krb5kdc[22652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041291, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:51.740700 22917 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:34:51.741688 22917 webserver.cc:460] Webserver started at http://127.18.125.130:46633/ using document root <none> and password file <none>
I20260806 18:34:51.742276 22917 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:51.742339 22917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:51.742537 22917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:51.745232 22917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "1ac14f594fb04b828e6ab7c5fe123ea5"
format_stamp: "Formatted at 2026-08-06 18:34:51 on dist-test-slave-8w9v"
server_key: "799c694d7b3c2be43470b5a83c294f62"
server_key_iv: "978de4d90f45d76bb467bceda56c6422"
server_key_version: "encryptionkey@0"
I20260806 18:34:51.746063 22917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "1ac14f594fb04b828e6ab7c5fe123ea5"
format_stamp: "Formatted at 2026-08-06 18:34:51 on dist-test-slave-8w9v"
server_key: "799c694d7b3c2be43470b5a83c294f62"
server_key_iv: "978de4d90f45d76bb467bceda56c6422"
server_key_version: "encryptionkey@0"
I20260806 18:34:51.758144 22917 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.006s sys 0.000s
I20260806 18:34:51.762279 22932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:51.763861 22917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:51.764009 22917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "1ac14f594fb04b828e6ab7c5fe123ea5"
format_stamp: "Formatted at 2026-08-06 18:34:51 on dist-test-slave-8w9v"
server_key: "799c694d7b3c2be43470b5a83c294f62"
server_key_iv: "978de4d90f45d76bb467bceda56c6422"
server_key_version: "encryptionkey@0"
I20260806 18:34:51.764228 22917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:51.782665 22917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:51.787470 22917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:51.787703 22917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:51.788609 22917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:51.789824 22917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:51.789896 22917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:51.789968 22917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:51.790004 22917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:51.804102 22917 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:34451
I20260806 18:34:51.804160 23051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:34451 every 8 connection(s)
I20260806 18:34:51.805605 22917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:51.810684 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 22917
I20260806 18:34:51.810819 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:51.811123 18934 external_mini_cluster.cc:1428] Setting key 53b64367511601ce1e5a9f8216036548
Aug 06 18:34:51 dist-test-slave-8w9v krb5kdc[22652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041291, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:34:51.827160 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:51.808090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:49321 (local address 127.18.125.190:42525)
0806 18:34:51.808264 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:51.808269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:51.809627 (+ 1358us) server_negotiation.cc:407] Connection header received
0806 18:34:51.811852 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:51.811860 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:51.811944 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:51.812036 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:51.814249 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.815437 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.817637 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.817903 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.818166 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:51.821927 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:51.821931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:51.821936 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:51.821974 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:51.823910 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:51.824744 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:51.824750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:51.824756 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:51.824860 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:51.825321 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:51.825327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:51.825332 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:51.825703 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:51.825827 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:51.826801 (+ 974us) server_negotiation.cc:299] Negotiation successful
0806 18:34:51.826973 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
I20260806 18:34:51.828552 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:51.808728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42525 (local address 127.18.125.130:49321)
0806 18:34:51.809406 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:51.809478 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:51.811483 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:51.812324 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:51.812340 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:51.813003 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:51.813899 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:51.813932 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.816180 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.816184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:51.817472 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:51.817486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:51.818210 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:51.818215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:51.818449 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:51.819299 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:51.819318 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:51.821646 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:51.824052 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:51.824059 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:51.824078 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:51.824598 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:51.824956 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:51.824961 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:51.824963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:51.825188 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:51.825836 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:51.825846 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:51.826247 (+ 401us) client_negotiation.cc:769] Sending connection context
0806 18:34:51.826733 (+ 486us) client_negotiation.cc:240] Negotiation successful
0806 18:34:51.827123 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":184,"threads_started":1}
I20260806 18:34:51.829998 23052 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42525
I20260806 18:34:51.830363 23052 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:51.831094 23052 heartbeater.cc:511] Master 127.18.125.190:42525 requested a full tablet report, sending...
I20260806 18:34:51.832365 22701 ts_manager.cc:194] Registered new tserver with Master: 1ac14f594fb04b828e6ab7c5fe123ea5 (127.18.125.130:34451)
I20260806 18:34:51.833235 22701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:49321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:34:51.848021 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42525
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:51.972347 23059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:51.972652 23059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:51.972700 23059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:51.977002 23059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:51.977108 23059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:51.977209 23059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:34:51.983269 23059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:51.984531 23059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:51.985632 23059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:51.998783 23065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:51.998950 23067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:51.998783 23064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:51.999531 23059 server_base.cc:1034] running on GCE node
I20260806 18:34:52.000070 23059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:52.000963 23059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:52.002177 23059 hybrid_clock.cc:648] HybridClock initialized: now 1786041292002155 us; error 41 us; skew 500 ppm
Aug 06 18:34:52 dist-test-slave-8w9v krb5kdc[22652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041292, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:52.005928 23059 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:34:52.007062 23059 webserver.cc:460] Webserver started at http://127.18.125.131:44995/ using document root <none> and password file <none>
I20260806 18:34:52.007866 23059 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:52.007982 23059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:52.008333 23059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:52.010775 23059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "19da55711b2c40efaa3c53fef43f3e5d"
format_stamp: "Formatted at 2026-08-06 18:34:52 on dist-test-slave-8w9v"
server_key: "17526f21a0e660a5da7acf684a969512"
server_key_iv: "d33de368131a36fd1ed563e78bf9ac1e"
server_key_version: "encryptionkey@0"
I20260806 18:34:52.011480 23059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "19da55711b2c40efaa3c53fef43f3e5d"
format_stamp: "Formatted at 2026-08-06 18:34:52 on dist-test-slave-8w9v"
server_key: "17526f21a0e660a5da7acf684a969512"
server_key_iv: "d33de368131a36fd1ed563e78bf9ac1e"
server_key_version: "encryptionkey@0"
I20260806 18:34:52.016635 23059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 18:34:52.019486 23074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:52.021013 23059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:52.021179 23059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "19da55711b2c40efaa3c53fef43f3e5d"
format_stamp: "Formatted at 2026-08-06 18:34:52 on dist-test-slave-8w9v"
server_key: "17526f21a0e660a5da7acf684a969512"
server_key_iv: "d33de368131a36fd1ed563e78bf9ac1e"
server_key_version: "encryptionkey@0"
I20260806 18:34:52.021325 23059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:52.039335 23059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:52.043107 23059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:52.043345 23059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:52.044028 23059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:52.045014 23059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:52.045112 23059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:52.045188 23059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:52.045241 23059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:52.055745 23059 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:35485
I20260806 18:34:52.055779 23193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:35485 every 8 connection(s)
I20260806 18:34:52.056787 23059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:52.066366 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 23059
I20260806 18:34:52.066560 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:52.066957 18934 external_mini_cluster.cc:1428] Setting key 3d78450b8acc4a8ff050e54260bcbf38
Aug 06 18:34:52 dist-test-slave-8w9v krb5kdc[22652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041292, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:52.073001 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:52.058832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:36707 (local address 127.18.125.190:42525)
0806 18:34:52.059018 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:52.059022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:52.060556 (+ 1534us) server_negotiation.cc:407] Connection header received
0806 18:34:52.062262 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:52.062266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:52.062333 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:52.062452 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:52.063773 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.064418 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.065687 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.065912 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.066024 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:52.069378 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:52.069381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:52.069383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:52.069409 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:52.070924 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:52.071464 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:52.071466 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:52.071467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:52.071540 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:52.071887 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:52.071889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:52.071890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:52.072067 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:52.072210 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:52.072759 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 18:34:52.072874 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:34:52.073915 23196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:52.059344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42525 (local address 127.18.125.131:36707)
0806 18:34:52.060297 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:52.060382 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:52.061991 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:52.062651 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:52.062659 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:52.063078 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:52.063598 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:52.063613 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.064567 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.064570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:52.065523 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:52.065533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.066117 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.066124 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:52.066295 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:52.067040 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:52.067073 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:52.069162 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:52.071049 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:52.071055 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:52.071069 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:52.071365 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:52.071628 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:52.071630 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:52.071631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:52.071798 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:52.072241 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:52.072247 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:52.072517 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 18:34:52.072718 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 18:34:52.073007 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":133,"threads_started":1}
I20260806 18:34:52.075246 23194 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42525
I20260806 18:34:52.075500 23194 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:52.075999 23194 heartbeater.cc:511] Master 127.18.125.190:42525 requested a full tablet report, sending...
I20260806 18:34:52.077126 22701 ts_manager.cc:194] Registered new tserver with Master: 19da55711b2c40efaa3c53fef43f3e5d (127.18.125.131:35485)
I20260806 18:34:52.077769 22701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:36707
I20260806 18:34:52.082433 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:34:52 dist-test-slave-8w9v krb5kdc[22652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:34:52 dist-test-slave-8w9v krb5kdc[22652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:34:52.114419 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:52.101848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.18.125.129:46097)
0806 18:34:52.102354 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:52.102361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:52.102393 (+ 32us) server_negotiation.cc:407] Connection header received
0806 18:34:52.102455 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:52.102460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:52.102681 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:52.102823 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:52.103877 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.104902 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.106153 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.106599 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.106700 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:52.110158 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:52.110170 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:52.110187 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:52.110220 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:52.112080 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:52.112613 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:52.112619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:52.112620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:52.112717 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:52.113148 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:52.113151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:52.113153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:52.113455 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:52.113642 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:52.113999 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 18:34:52.114219 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":128,"threads_started":1}
Aug 06 18:34:52 dist-test-slave-8w9v krb5kdc[22652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:52.127962 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:52.118668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49092 (local address 127.18.125.190:42525)
0806 18:34:52.118844 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:52.118848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:52.118863 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:34:52.118981 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:52.118984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:52.119029 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:52.119100 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:52.119739 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.120410 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.121267 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:52.121500 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:52.121596 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:52.124309 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:52.124313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:52.124316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:52.124343 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:52.125843 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:52.126283 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:52.126286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:52.126289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:52.126374 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:52.126675 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:52.126678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:52.126679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:52.126937 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:52.127067 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:52.127667 (+ 600us) server_negotiation.cc:299] Negotiation successful
0806 18:34:52.127830 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 18:34:52.128562 22701 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49092
I20260806 18:34:52.129532 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 22775
I20260806 18:34:52.137848 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 22917
I20260806 18:34:52.145210 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 23059
I20260806 18:34:52.155401 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 22665
2026-08-06T18:34:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2341 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:52 dist-test-slave-8w9v krb5kdc[23208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:52 dist-test-slave-8w9v krb5kdc[23208](info): set up 2 sockets
Aug 06 18:34:52 dist-test-slave-8w9v krb5kdc[23208](info): commencing operation
krb5kdc: starting...
W20260806 18:34:53.258854 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:34:53 dist-test-slave-8w9v krb5kdc[23208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:34:53.393988 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:46771
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:46771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:53.515236 23222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:53.515552 23222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:53.515600 23222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:53.519824 23222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:53.519917 23222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:53.519939 23222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:53.519959 23222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:53.519979 23222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:53.525588 23222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:46771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:46771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:53.526738 23222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:53.527807 23222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:53.537237 23230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:53.537194 23228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:53.537348 23222 server_base.cc:1034] running on GCE node
W20260806 18:34:53.537155 23227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:53.538081 23222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:53.539306 23222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:53.540508 23222 hybrid_clock.cc:648] HybridClock initialized: now 1786041293540495 us; error 27 us; skew 500 ppm
Aug 06 18:34:53 dist-test-slave-8w9v krb5kdc[23208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041293, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:53.550424 23222 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:34:53.551314 23222 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:53.551419 23222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:53.551664 23222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:53.554116 23222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "1c88c9dbfc9f4335b20d3021a52d19fa"
format_stamp: "Formatted at 2026-08-06 18:34:53 on dist-test-slave-8w9v"
server_key: "09182a86898877dce3df01c113a4cde0"
server_key_iv: "ff0bb5d20f18fb15bf54506fe44c5f5a"
server_key_version: "encryptionkey@0"
I20260806 18:34:53.554714 23222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "1c88c9dbfc9f4335b20d3021a52d19fa"
format_stamp: "Formatted at 2026-08-06 18:34:53 on dist-test-slave-8w9v"
server_key: "09182a86898877dce3df01c113a4cde0"
server_key_iv: "ff0bb5d20f18fb15bf54506fe44c5f5a"
server_key_version: "encryptionkey@0"
I20260806 18:34:53.558679 23222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 18:34:53.561390 23237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:53.562672 23222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:34:53.562834 23222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "1c88c9dbfc9f4335b20d3021a52d19fa"
format_stamp: "Formatted at 2026-08-06 18:34:53 on dist-test-slave-8w9v"
server_key: "09182a86898877dce3df01c113a4cde0"
server_key_iv: "ff0bb5d20f18fb15bf54506fe44c5f5a"
server_key_version: "encryptionkey@0"
I20260806 18:34:53.562955 23222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:53.587878 23222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:53.591591 23222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:53.591812 23222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:53.599066 23222 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:46771
I20260806 18:34:53.599339 23290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:46771 every 8 connection(s)
I20260806 18:34:53.599771 23222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:53.602134 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 23222
I20260806 18:34:53.602332 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:53.602663 18934 external_mini_cluster.cc:1428] Setting key 233200aca3a25df6c9f52beb398ee7ca
I20260806 18:34:53.602837 23291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:53.610368 23291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa: Bootstrap starting.
Aug 06 18:34:53 dist-test-slave-8w9v krb5kdc[23208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:53.613505 23291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:53.614552 23291 log.cc:824] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:53.617166 23291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa: No bootstrap required, opened a new log
I20260806 18:34:53.618899 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:53.604297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38660 (local address 127.18.125.190:46771)
0806 18:34:53.604731 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:53.604748 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:53.604785 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:34:53.605712 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:53.605732 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:53.606078 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:53.606365 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:53.607332 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.608458 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.609644 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.610146 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.610270 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:53.613369 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:53.613378 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:53.613401 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:53.613446 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:53.615422 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:53.615925 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:53.615931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:53.615937 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:53.616091 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:53.616463 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:53.616466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:53.616468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:53.616996 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:53.617184 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:53.617601 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 18:34:53.617919 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":148,"threads_started":1}
I20260806 18:34:53.621693 23291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c88c9dbfc9f4335b20d3021a52d19fa" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46771 } }
I20260806 18:34:53.621938 23291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:53.621975 23291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c88c9dbfc9f4335b20d3021a52d19fa, State: Initialized, Role: FOLLOWER
I20260806 18:34:53.622355 23291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c88c9dbfc9f4335b20d3021a52d19fa" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46771 } }
I20260806 18:34:53.622479 23291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:53.622526 23291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:53.622601 23291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:53.623777 23291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c88c9dbfc9f4335b20d3021a52d19fa" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46771 } }
I20260806 18:34:53.624095 23291 leader_election.cc:302] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c88c9dbfc9f4335b20d3021a52d19fa; no voters:
I20260806 18:34:53.624456 23291 leader_election.cc:288] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:53.624608 23296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:53.624897 23296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [term 1 LEADER]: Becoming Leader. State: Replica: 1c88c9dbfc9f4335b20d3021a52d19fa, State: Running, Role: LEADER
I20260806 18:34:53.625227 23296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c88c9dbfc9f4335b20d3021a52d19fa" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46771 } }
I20260806 18:34:53.625810 23291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:53.626993 23298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c88c9dbfc9f4335b20d3021a52d19fa. Latest consensus state: current_term: 1 leader_uuid: "1c88c9dbfc9f4335b20d3021a52d19fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c88c9dbfc9f4335b20d3021a52d19fa" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46771 } } }
I20260806 18:34:53.627159 23298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:53.626993 23297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c88c9dbfc9f4335b20d3021a52d19fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c88c9dbfc9f4335b20d3021a52d19fa" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 46771 } } }
I20260806 18:34:53.627511 23297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:53.627910 23306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:53.630865 23306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:53.637538 23306 catalog_manager.cc:1347] Generated new cluster ID: 3276fc290ebb4ac68dd3b38f0f1720ba
I20260806 18:34:53.637621 23306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:53.658567 23306 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:53.659610 23306 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:53.672870 23306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa: Generated new TSK 0
I20260806 18:34:53.673671 23306 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:34:53.708500 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46771
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:53.838735 23318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:53.839077 23318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:53.839129 23318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:53.843307 23318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:53.843441 23318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:53.843556 23318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:34:53.849284 23318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:53.850581 23318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:53.851752 23318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:34:53.862320 23318 server_base.cc:1034] running on GCE node
W20260806 18:34:53.862329 23323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:53.862329 23324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:53.862320 23326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:53.863122 23318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:53.863788 23318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:53.865015 23318 hybrid_clock.cc:648] HybridClock initialized: now 1786041293864974 us; error 71 us; skew 500 ppm
Aug 06 18:34:53 dist-test-slave-8w9v krb5kdc[23208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041293, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:53.868438 23318 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:34:53.869273 23318 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:53.869349 23318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:53.869578 23318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:53.871821 23318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "7bfbab1fd3da4cd08631d4a7e58809c1"
format_stamp: "Formatted at 2026-08-06 18:34:53 on dist-test-slave-8w9v"
server_key: "6d725243791acb192786887fa2baa103"
server_key_iv: "0906390285b0b222dc0ccc185eb776a1"
server_key_version: "encryptionkey@0"
I20260806 18:34:53.872514 23318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "7bfbab1fd3da4cd08631d4a7e58809c1"
format_stamp: "Formatted at 2026-08-06 18:34:53 on dist-test-slave-8w9v"
server_key: "6d725243791acb192786887fa2baa103"
server_key_iv: "0906390285b0b222dc0ccc185eb776a1"
server_key_version: "encryptionkey@0"
I20260806 18:34:53.876791 23318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:34:53.879590 23332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:53.881088 23318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:53.881232 23318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "7bfbab1fd3da4cd08631d4a7e58809c1"
format_stamp: "Formatted at 2026-08-06 18:34:53 on dist-test-slave-8w9v"
server_key: "6d725243791acb192786887fa2baa103"
server_key_iv: "0906390285b0b222dc0ccc185eb776a1"
server_key_version: "encryptionkey@0"
I20260806 18:34:53.881366 23318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:53.922966 23318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:53.926832 23318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:53.927048 23318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:53.927744 23318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:53.928737 23318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:53.928788 23318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:53.928834 23318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:53.928890 23318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:53.939137 23318 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:46591
I20260806 18:34:53.939158 23453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:46591 every 8 connection(s)
I20260806 18:34:53.939857 23318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:53.947423 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 23318
I20260806 18:34:53.947537 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:53.947880 18934 external_mini_cluster.cc:1428] Setting key 475878695330e1330daca25588908b29
Aug 06 18:34:53 dist-test-slave-8w9v krb5kdc[23208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041293, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:53.955569 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:53.941996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:56205 (local address 127.18.125.190:46771)
0806 18:34:53.942161 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:53.942165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:53.943130 (+ 965us) server_negotiation.cc:407] Connection header received
0806 18:34:53.944677 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:53.944680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:53.944746 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:53.944839 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:53.946102 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.946675 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.947976 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.948324 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.948444 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:53.951431 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:53.951437 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:53.951445 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:53.951469 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:53.953002 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:53.953710 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:53.953714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:53.953718 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:53.953846 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:53.954290 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:53.954293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:53.954294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:53.954506 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:53.954597 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:53.955317 (+ 720us) server_negotiation.cc:299] Negotiation successful
0806 18:34:53.955443 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:34:53.956974 23456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:53.942383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:46771 (local address 127.18.125.129:56205)
0806 18:34:53.942921 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:53.942993 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:53.944448 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:53.945013 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:53.945021 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:53.945410 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:53.945917 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:53.945949 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.946817 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.946821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:53.947833 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:53.947844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:53.948492 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:53.948496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:53.948670 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:53.949463 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:53.949488 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:53.951256 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:53.953171 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:53.953181 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:53.953200 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:53.953570 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:53.953955 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:53.953961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:53.953966 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:53.954164 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:53.954632 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:53.954642 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:53.955022 (+ 380us) client_negotiation.cc:769] Sending connection context
0806 18:34:53.955269 (+ 247us) client_negotiation.cc:240] Negotiation successful
0806 18:34:53.955673 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":132,"threads_started":1}
I20260806 18:34:53.958364 23454 heartbeater.cc:347] Connected to a master server at 127.18.125.190:46771
I20260806 18:34:53.958626 23454 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:53.959112 23454 heartbeater.cc:511] Master 127.18.125.190:46771 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:34:53.960843 23255 ts_manager.cc:194] Registered new tserver with Master: 7bfbab1fd3da4cd08631d4a7e58809c1 (127.18.125.129:46591)
I20260806 18:34:53.962077 23255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:56205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:34:53.980523 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46771
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:54.101183 23459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:54.101524 23459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:54.101606 23459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:54.105803 23459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:54.105930 23459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:54.106038 23459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:34:54.111716 23459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:54.112998 23459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:54.114104 23459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:54.124071 23464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:54.124071 23465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:54.124073 23467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:54.124564 23459 server_base.cc:1034] running on GCE node
I20260806 18:34:54.125006 23459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:54.125679 23459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:54.126905 23459 hybrid_clock.cc:648] HybridClock initialized: now 1786041294126822 us; error 89 us; skew 500 ppm
Aug 06 18:34:54 dist-test-slave-8w9v krb5kdc[23208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041294, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:54.130239 23459 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:34:54.131093 23459 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:54.131196 23459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:54.131451 23459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:54.133734 23459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "8ce2808e951844d3ae5567f8b314333c"
format_stamp: "Formatted at 2026-08-06 18:34:54 on dist-test-slave-8w9v"
server_key: "45719c665872ce623a307386cffc629f"
server_key_iv: "9e162301344b6347d99763972d1aefcd"
server_key_version: "encryptionkey@0"
I20260806 18:34:54.134368 23459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "8ce2808e951844d3ae5567f8b314333c"
format_stamp: "Formatted at 2026-08-06 18:34:54 on dist-test-slave-8w9v"
server_key: "45719c665872ce623a307386cffc629f"
server_key_iv: "9e162301344b6347d99763972d1aefcd"
server_key_version: "encryptionkey@0"
I20260806 18:34:54.138782 23459 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:54.141677 23473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:54.143102 23459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:34:54.143258 23459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "8ce2808e951844d3ae5567f8b314333c"
format_stamp: "Formatted at 2026-08-06 18:34:54 on dist-test-slave-8w9v"
server_key: "45719c665872ce623a307386cffc629f"
server_key_iv: "9e162301344b6347d99763972d1aefcd"
server_key_version: "encryptionkey@0"
I20260806 18:34:54.143399 23459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:54.157894 23459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:54.161406 23459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:54.161635 23459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:54.162420 23459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:54.163359 23459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:54.163436 23459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:54.163512 23459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:54.163554 23459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:54.173207 23459 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:45791
I20260806 18:34:54.173250 23594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:45791 every 8 connection(s)
I20260806 18:34:54.173898 23459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:54.178149 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 23459
I20260806 18:34:54.178280 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:54.178685 18934 external_mini_cluster.cc:1428] Setting key 6f5bb64c7258e448101a59ace5d648b5
Aug 06 18:34:54 dist-test-slave-8w9v krb5kdc[23208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041294, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:34:54.191176 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.175783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:46429 (local address 127.18.125.190:46771)
0806 18:34:54.175921 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.175925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.176944 (+ 1019us) server_negotiation.cc:407] Connection header received
0806 18:34:54.178648 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.178651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.178700 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.178817 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:54.180552 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.181098 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.183129 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.183539 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.183676 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.187146 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:54.187149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:54.187151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:54.187179 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:54.188843 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:54.189423 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:54.189426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:54.189427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:54.189519 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:54.189899 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:54.189902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:54.189904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:54.190110 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:54.190225 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:54.190854 (+ 629us) server_negotiation.cc:299] Negotiation successful
0806 18:34:54.190996 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:34:54.192113 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.176290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:46771 (local address 127.18.125.130:46429)
0806 18:34:54.176777 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:54.176825 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:54.178391 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:54.179053 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:54.179064 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:54.179626 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:54.180375 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.180400 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.181255 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.181262 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.182963 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.182974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.183728 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.183732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.183917 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.184641 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:54.184678 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:54.186926 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:54.188980 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:54.188987 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:54.189005 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:54.189285 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:54.189625 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:54.189628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:54.189630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:54.189788 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:54.190225 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:54.190233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:54.190553 (+ 320us) client_negotiation.cc:769] Sending connection context
0806 18:34:54.190788 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 18:34:54.191157 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":103,"threads_started":1}
I20260806 18:34:54.193517 23595 heartbeater.cc:347] Connected to a master server at 127.18.125.190:46771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
I20260806 18:34:54.193856 23595 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:54.194435 23595 heartbeater.cc:511] Master 127.18.125.190:46771 requested a full tablet report, sending...
I20260806 18:34:54.195730 23255 ts_manager.cc:194] Registered new tserver with Master: 8ce2808e951844d3ae5567f8b314333c (127.18.125.130:45791)
I20260806 18:34:54.196762 23255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:46429
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:34:54.212515 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46771
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:54.343775 23601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:54.344084 23601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:54.344202 23601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:54.348629 23601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:54.348709 23601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:54.348778 23601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:34:54.354707 23601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:46771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:54.355871 23601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:54.357005 23601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:54.367307 23606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:54.367327 23607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:54.367331 23609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:54.367753 23601 server_base.cc:1034] running on GCE node
I20260806 18:34:54.368341 23601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:54.369050 23601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:54.370249 23601 hybrid_clock.cc:648] HybridClock initialized: now 1786041294370221 us; error 45 us; skew 500 ppm
Aug 06 18:34:54 dist-test-slave-8w9v krb5kdc[23208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041294, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:54.373615 23601 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:34:54.374569 23601 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:54.374646 23601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:54.374894 23601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:54.377316 23601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "08f53e3bfff74ff4952e2cfe43750601"
format_stamp: "Formatted at 2026-08-06 18:34:54 on dist-test-slave-8w9v"
server_key: "bb07129f5801d8a225a0b472d495b9b2"
server_key_iv: "cdcc3a12870f8d8b902a511995fda3fb"
server_key_version: "encryptionkey@0"
I20260806 18:34:54.378002 23601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "08f53e3bfff74ff4952e2cfe43750601"
format_stamp: "Formatted at 2026-08-06 18:34:54 on dist-test-slave-8w9v"
server_key: "bb07129f5801d8a225a0b472d495b9b2"
server_key_iv: "cdcc3a12870f8d8b902a511995fda3fb"
server_key_version: "encryptionkey@0"
I20260806 18:34:54.382197 23601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:54.384992 23616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:54.386514 23601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:54.386656 23601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "08f53e3bfff74ff4952e2cfe43750601"
format_stamp: "Formatted at 2026-08-06 18:34:54 on dist-test-slave-8w9v"
server_key: "bb07129f5801d8a225a0b472d495b9b2"
server_key_iv: "cdcc3a12870f8d8b902a511995fda3fb"
server_key_version: "encryptionkey@0"
I20260806 18:34:54.386787 23601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:54.413976 23601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:54.417696 23601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:54.417959 23601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:54.418614 23601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:54.419509 23601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:54.419559 23601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:54.419626 23601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:54.419661 23601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:54.429587 23601 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:43225
I20260806 18:34:54.429605 23737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:43225 every 8 connection(s)
I20260806 18:34:54.430234 23601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:54.430611 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 23601
I20260806 18:34:54.430742 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:54.431085 18934 external_mini_cluster.cc:1428] Setting key 912d38b5722bf2880f8a9e58febf9398
Aug 06 18:34:54 dist-test-slave-8w9v krb5kdc[23208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041294, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:54.444443 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.432116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:38635 (local address 127.18.125.190:46771)
0806 18:34:54.432302 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.432307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.433374 (+ 1067us) server_negotiation.cc:407] Connection header received
0806 18:34:54.434794 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.434797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.434847 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.434922 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:54.436192 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.436758 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.437960 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.438181 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.438266 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.441051 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:54.441054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:54.441057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:54.441082 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:54.442309 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:54.442823 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:54.442825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:54.442826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:54.442904 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:54.443276 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:54.443279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:54.443280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:54.443485 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:54.443567 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:54.444101 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 18:34:54.444258 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 18:34:54.445551 23740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.432682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:46771 (local address 127.18.125.131:38635)
0806 18:34:54.433209 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:54.433265 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:54.434584 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:54.435097 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:54.435103 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:54.435503 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:54.436012 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.436026 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.436907 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.436910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.437824 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.437835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.438334 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.438337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.438514 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.439212 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:54.439230 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:54.440853 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:54.442456 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:54.442461 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:54.442474 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:54.442730 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:54.442993 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:54.442995 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:54.442997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:54.443167 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:54.443582 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:54.443587 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:54.443835 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 18:34:54.444022 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 18:34:54.444440 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":149,"threads_started":1}
I20260806 18:34:54.446653 23738 heartbeater.cc:347] Connected to a master server at 127.18.125.190:46771
I20260806 18:34:54.446931 23738 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:54.447448 23738 heartbeater.cc:511] Master 127.18.125.190:46771 requested a full tablet report, sending...
I20260806 18:34:54.448606 23255 ts_manager.cc:194] Registered new tserver with Master: 08f53e3bfff74ff4952e2cfe43750601 (127.18.125.131:43225)
I20260806 18:34:54.449187 23255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:38635
I20260806 18:34:54.457448 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:54.467253 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.459595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38664 (local address 127.18.125.190:46771)
0806 18:34:54.459747 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.459754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.459927 (+ 173us) server_negotiation.cc:407] Connection header received
0806 18:34:54.460097 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.460100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.460191 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.460275 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:54.461022 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.461575 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.462407 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.462629 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.462736 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.464173 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:54.464176 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:54.464179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:54.464205 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:54.465414 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:54.465870 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:54.465873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:54.465879 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:54.466014 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:54.466292 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:54.466296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:54.466298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:54.466543 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:54.466691 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:54.466955 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 18:34:54.467077 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:34:54.470157 23255 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:54.473361 23255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:54.488163 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.482318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:45791 (local address 127.0.0.1:52808)
0806 18:34:54.482626 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:54.482674 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:54.482799 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:54.483913 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:54.483919 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:54.483943 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:54.484060 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.484064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.485317 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.485322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.486690 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.486700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.487564 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.487567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.487673 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.487716 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 18:34:54.487837 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 18:34:54.487958 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260806 18:34:54.488188 23752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.482500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.18.125.130:45791)
0806 18:34:54.483168 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.483180 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.483208 (+ 28us) server_negotiation.cc:407] Connection header received
0806 18:34:54.483292 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.483297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.483504 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.483671 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:54.484205 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.485185 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.486823 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.487430 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.487565 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.487710 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:54.487883 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 18:34:54.487992 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":169,"threads_started":1}
I20260806 18:34:54.488952 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.483016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:43225 (local address 127.0.0.1:44200)
0806 18:34:54.483755 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:54.483770 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:54.483861 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:54.484608 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:54.484613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:54.484636 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:54.484731 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.484737 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.486156 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.486159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.487297 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.487317 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.488544 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.488548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.488635 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.488658 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:34:54.488741 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:34:54.488820 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":90,"threads_started":1}
I20260806 18:34:54.489030 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.483381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.18.125.131:43225)
0806 18:34:54.484011 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.484023 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.484042 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:34:54.484105 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.484109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.484291 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.484433 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:54.484905 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.486017 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.487453 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.488387 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.488546 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.488665 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:54.488777 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 18:34:54.488866 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":153,"threads_started":1}
I20260806 18:34:54.489202 23754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.482642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:46591 (local address 127.0.0.1:35228)
0806 18:34:54.483727 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:54.483741 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:54.483852 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:54.484411 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:54.484417 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:54.484439 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:54.484540 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.484546 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.486032 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.486038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.487428 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.487435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.488715 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.488718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.488813 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.488842 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:34:54.488914 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:34:54.488991 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"thread_start_us":99,"threads_started":1}
I20260806 18:34:54.489730 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.483103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35228 (local address 127.18.125.129:46591)
0806 18:34:54.483585 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.483593 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.483960 (+ 367us) server_negotiation.cc:407] Connection header received
0806 18:34:54.484047 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.484052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.484266 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.484425 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:54.484664 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.485894 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.487523 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.488396 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.488518 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.488648 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:54.489486 (+ 838us) server_negotiation.cc:299] Negotiation successful
0806 18:34:54.489591 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":97,"threads_started":1}
I20260806 18:34:54.491932 23526 tablet_service.cc:1467] Processing CreateTablet for tablet 5d760af0344d40c9b603b7db4b4eccd5 (DEFAULT_TABLE table=test-table [id=4925b00b4baa4f92a12d92da0c5eca56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:54.493078 23526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d760af0344d40c9b603b7db4b4eccd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:54.492688 23382 tablet_service.cc:1467] Processing CreateTablet for tablet 5d760af0344d40c9b603b7db4b4eccd5 (DEFAULT_TABLE table=test-table [id=4925b00b4baa4f92a12d92da0c5eca56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:54.492691 23667 tablet_service.cc:1467] Processing CreateTablet for tablet 5d760af0344d40c9b603b7db4b4eccd5 (DEFAULT_TABLE table=test-table [id=4925b00b4baa4f92a12d92da0c5eca56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:54.493675 23382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d760af0344d40c9b603b7db4b4eccd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:54.493692 23667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d760af0344d40c9b603b7db4b4eccd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:54.499961 23757 tablet_bootstrap.cc:492] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c: Bootstrap starting.
I20260806 18:34:54.501479 23758 tablet_bootstrap.cc:492] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1: Bootstrap starting.
I20260806 18:34:54.502075 23759 tablet_bootstrap.cc:492] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601: Bootstrap starting.
I20260806 18:34:54.502676 23757 tablet_bootstrap.cc:654] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:54.503930 23759 tablet_bootstrap.cc:654] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:54.504040 23758 tablet_bootstrap.cc:654] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:54.504058 23757 log.cc:824] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:54.504940 23758 log.cc:824] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:54.505299 23759 log.cc:824] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:54.506564 23757 tablet_bootstrap.cc:492] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c: No bootstrap required, opened a new log
I20260806 18:34:54.506791 23757 ts_tablet_manager.cc:1397] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:34:54.506966 23758 tablet_bootstrap.cc:492] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1: No bootstrap required, opened a new log
I20260806 18:34:54.507154 23758 ts_tablet_manager.cc:1397] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:34:54.508214 23759 tablet_bootstrap.cc:492] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601: No bootstrap required, opened a new log
I20260806 18:34:54.508432 23759 ts_tablet_manager.cc:1397] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 18:34:54.511226 23759 raft_consensus.cc:348] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce2808e951844d3ae5567f8b314333c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45791 } } peers { permanent_uuid: "08f53e3bfff74ff4952e2cfe43750601" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43225 } } peers { permanent_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 46591 } }
I20260806 18:34:54.511224 23757 raft_consensus.cc:348] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce2808e951844d3ae5567f8b314333c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45791 } } peers { permanent_uuid: "08f53e3bfff74ff4952e2cfe43750601" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43225 } } peers { permanent_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 46591 } }
I20260806 18:34:54.511518 23757 raft_consensus.cc:374] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:54.511521 23759 raft_consensus.cc:374] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:54.511585 23759 raft_consensus.cc:729] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08f53e3bfff74ff4952e2cfe43750601, State: Initialized, Role: FOLLOWER
I20260806 18:34:54.511617 23757 raft_consensus.cc:729] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce2808e951844d3ae5567f8b314333c, State: Initialized, Role: FOLLOWER
I20260806 18:34:54.511520 23758 raft_consensus.cc:348] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce2808e951844d3ae5567f8b314333c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45791 } } peers { permanent_uuid: "08f53e3bfff74ff4952e2cfe43750601" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43225 } } peers { permanent_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 46591 } }
I20260806 18:34:54.511792 23758 raft_consensus.cc:374] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:54.511870 23758 raft_consensus.cc:729] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bfbab1fd3da4cd08631d4a7e58809c1, State: Initialized, Role: FOLLOWER
I20260806 18:34:54.512167 23757 consensus_queue.cc:260] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce2808e951844d3ae5567f8b314333c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45791 } } peers { permanent_uuid: "08f53e3bfff74ff4952e2cfe43750601" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43225 } } peers { permanent_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 46591 } }
I20260806 18:34:54.512168 23759 consensus_queue.cc:260] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce2808e951844d3ae5567f8b314333c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45791 } } peers { permanent_uuid: "08f53e3bfff74ff4952e2cfe43750601" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43225 } } peers { permanent_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 46591 } }
I20260806 18:34:54.512331 23758 consensus_queue.cc:260] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce2808e951844d3ae5567f8b314333c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45791 } } peers { permanent_uuid: "08f53e3bfff74ff4952e2cfe43750601" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43225 } } peers { permanent_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 46591 } }
I20260806 18:34:54.513291 23758 ts_tablet_manager.cc:1428] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:34:54.513291 23757 ts_tablet_manager.cc:1428] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:34:54.513293 23759 ts_tablet_manager.cc:1428] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:34:54.513618 23738 heartbeater.cc:503] Master 127.18.125.190:46771 was elected leader, sending a full tablet report...
I20260806 18:34:54.513674 23454 heartbeater.cc:503] Master 127.18.125.190:46771 was elected leader, sending a full tablet report...
I20260806 18:34:54.514772 23595 heartbeater.cc:503] Master 127.18.125.190:46771 was elected leader, sending a full tablet report...
I20260806 18:34:54.610843 23765 raft_consensus.cc:482] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:54.611116 23765 raft_consensus.cc:504] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce2808e951844d3ae5567f8b314333c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45791 } } peers { permanent_uuid: "08f53e3bfff74ff4952e2cfe43750601" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43225 } } peers { permanent_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 46591 } }
I20260806 18:34:54.612361 23765 leader_election.cc:288] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ce2808e951844d3ae5567f8b314333c (127.18.125.130:45791), 08f53e3bfff74ff4952e2cfe43750601 (127.18.125.131:43225)
I20260806 18:34:54.616546 23752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.612833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:49815 (local address 127.18.125.130:45791)
0806 18:34:54.613008 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.613013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.613178 (+ 165us) server_negotiation.cc:407] Connection header received
0806 18:34:54.613410 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.613414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.613466 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.613554 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:54.613889 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.614423 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.615526 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.615939 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.616085 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.616160 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:54.616354 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 18:34:54.616427 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:34:54.616577 23766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.612638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:45791 (local address 127.18.125.129:49815)
0806 18:34:54.613058 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:54.613079 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:54.613273 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:54.613570 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:54.613573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:54.613591 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:54.613766 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.613772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.614543 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.614548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.615390 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.615400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.616078 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.616082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.616214 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.616243 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:34:54.616336 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:34:54.616408 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"spinlock_wait_cycles":35200,"thread_start_us":174,"threads_started":1}
I20260806 18:34:54.616773 23756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.613294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:56295 (local address 127.18.125.131:43225)
0806 18:34:54.613466 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.613471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.613619 (+ 148us) server_negotiation.cc:407] Connection header received
0806 18:34:54.613805 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.613808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.613853 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.613943 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:34:54.614211 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.614712 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.615735 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.616210 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.616338 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.616401 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:54.616590 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 18:34:54.616662 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 18:34:54.616797 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.613106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:43225 (local address 127.18.125.129:56295)
0806 18:34:54.613522 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:54.613533 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:54.613685 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:54.613955 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:54.613958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:54.613974 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:34:54.614097 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.614103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.614830 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.614835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.615602 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:54.615611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.616333 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.616336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:54.616448 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.616479 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:34:54.616572 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:34:54.616646 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"spinlock_wait_cycles":768,"thread_start_us":135,"threads_started":1}
I20260806 18:34:54.617434 23688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d760af0344d40c9b603b7db4b4eccd5" candidate_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08f53e3bfff74ff4952e2cfe43750601" is_pre_election: true
I20260806 18:34:54.617432 23547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d760af0344d40c9b603b7db4b4eccd5" candidate_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce2808e951844d3ae5567f8b314333c" is_pre_election: true
I20260806 18:34:54.617746 23688 raft_consensus.cc:2454] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bfbab1fd3da4cd08631d4a7e58809c1 in term 0.
I20260806 18:34:54.617740 23547 raft_consensus.cc:2454] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bfbab1fd3da4cd08631d4a7e58809c1 in term 0.
I20260806 18:34:54.618259 23333 leader_election.cc:302] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bfbab1fd3da4cd08631d4a7e58809c1, 8ce2808e951844d3ae5567f8b314333c; no voters:
I20260806 18:34:54.618580 23765 raft_consensus.cc:2781] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:54.618681 23765 raft_consensus.cc:482] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:54.618783 23765 raft_consensus.cc:3037] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:54.619894 23765 raft_consensus.cc:504] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce2808e951844d3ae5567f8b314333c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45791 } } peers { permanent_uuid: "08f53e3bfff74ff4952e2cfe43750601" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43225 } } peers { permanent_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 46591 } }
I20260806 18:34:54.620292 23765 leader_election.cc:288] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [CANDIDATE]: Term 1 election: Requested vote from peers 8ce2808e951844d3ae5567f8b314333c (127.18.125.130:45791), 08f53e3bfff74ff4952e2cfe43750601 (127.18.125.131:43225)
I20260806 18:34:54.620780 23547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d760af0344d40c9b603b7db4b4eccd5" candidate_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce2808e951844d3ae5567f8b314333c"
I20260806 18:34:54.620823 23688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d760af0344d40c9b603b7db4b4eccd5" candidate_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08f53e3bfff74ff4952e2cfe43750601"
I20260806 18:34:54.620939 23547 raft_consensus.cc:3037] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:54.620963 23688 raft_consensus.cc:3037] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:54.622066 23547 raft_consensus.cc:2454] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bfbab1fd3da4cd08631d4a7e58809c1 in term 1.
I20260806 18:34:54.622066 23688 raft_consensus.cc:2454] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bfbab1fd3da4cd08631d4a7e58809c1 in term 1.
I20260806 18:34:54.622426 23333 leader_election.cc:302] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bfbab1fd3da4cd08631d4a7e58809c1, 8ce2808e951844d3ae5567f8b314333c; no voters:
I20260806 18:34:54.622638 23765 raft_consensus.cc:2781] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:54.622902 23765 raft_consensus.cc:686] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [term 1 LEADER]: Becoming Leader. State: Replica: 7bfbab1fd3da4cd08631d4a7e58809c1, State: Running, Role: LEADER
I20260806 18:34:54.623260 23765 consensus_queue.cc:237] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce2808e951844d3ae5567f8b314333c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45791 } } peers { permanent_uuid: "08f53e3bfff74ff4952e2cfe43750601" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43225 } } peers { permanent_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 46591 } }
I20260806 18:34:54.627251 23253 catalog_manager.cc:5495] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bfbab1fd3da4cd08631d4a7e58809c1 (127.18.125.129). New cstate: current_term: 1 leader_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce2808e951844d3ae5567f8b314333c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08f53e3bfff74ff4952e2cfe43750601" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bfbab1fd3da4cd08631d4a7e58809c1" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 46591 } health_report { overall_health: HEALTHY } } }
I20260806 18:34:54.644037 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.640331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35232 (local address 127.18.125.129:46591)
0806 18:34:54.640492 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.640497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.640513 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:34:54.640647 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.640650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.640709 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.640843 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:34:54.641114 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.641637 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.642502 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:54.642732 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:54.642860 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:54.643188 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:34:54.643628 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:34:54.643765 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:54.643819 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 18:34:54.643880 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:34:54.650460 23547 raft_consensus.cc:1261] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c [term 1 FOLLOWER]: Refusing update from remote peer 7bfbab1fd3da4cd08631d4a7e58809c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:54.650460 23688 raft_consensus.cc:1261] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601 [term 1 FOLLOWER]: Refusing update from remote peer 7bfbab1fd3da4cd08631d4a7e58809c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:54.651229 23768 consensus_queue.cc:1035] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08f53e3bfff74ff4952e2cfe43750601" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:54.651429 23765 consensus_queue.cc:1035] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ce2808e951844d3ae5567f8b314333c" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:54.661574 23773 mvcc.cc:205] Tried to move back new op lower bound from 7315625142880403456 to 7315625142779957248. Current Snapshot: MvccSnapshot[applied={T|T < 7315625142880403456}]
I20260806 18:34:54.662050 23772 mvcc.cc:205] Tried to move back new op lower bound from 7315625142880403456 to 7315625142779957248. Current Snapshot: MvccSnapshot[applied={T|T < 7315625142880403456}]
I20260806 18:34:54.669337 23254 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:54.669502 23254 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:54.672236 23254 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c88c9dbfc9f4335b20d3021a52d19fa: Sending DeleteTablet for 3 replicas of tablet 5d760af0344d40c9b603b7db4b4eccd5
I20260806 18:34:54.673102 23667 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d760af0344d40c9b603b7db4b4eccd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:54 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:44200
I20260806 18:34:54.673074 23382 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d760af0344d40c9b603b7db4b4eccd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:54 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:35228
I20260806 18:34:54.673640 23780 tablet_replica.cc:330] T 5d760af0344d40c9b603b7db4b4eccd5 P 7bfbab1fd3da4cd08631d4a7e58809c1: stopping tablet replica
I20260806 18:34:54.673671 23526 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d760af0344d40c9b603b7db4b4eccd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:54 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:52808
I20260806 18:34:54.673908 23781 tablet_replica.cc:330] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601: stopping tablet replica
I20260806 18:34:54.674242 23781 raft_consensus.cc:2229] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:54.674261 23782 tablet_replica.cc:330] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c: stopping tablet replica
I20260806 18:34:54.674360 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 23318
I20260806 18:34:54.674590 23781 raft_consensus.cc:2258] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:54.676503 23782 raft_consensus.cc:2229] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:54.676802 23782 raft_consensus.cc:2258] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:54.676947 23781 ts_tablet_manager.cc:1905] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:54.678690 23782 ts_tablet_manager.cc:1905] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:54.680637 23781 ts_tablet_manager.cc:1918] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:54.680755 23781 log.cc:1196] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/5d760af0344d40c9b603b7db4b4eccd5
I20260806 18:34:54.681149 23781 ts_tablet_manager.cc:1939] T 5d760af0344d40c9b603b7db4b4eccd5 P 08f53e3bfff74ff4952e2cfe43750601: Deleting consensus metadata
I20260806 18:34:54.681301 23782 ts_tablet_manager.cc:1918] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:54.681432 23782 log.cc:1196] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/5d760af0344d40c9b603b7db4b4eccd5
I20260806 18:34:54.681792 23782 ts_tablet_manager.cc:1939] T 5d760af0344d40c9b603b7db4b4eccd5 P 8ce2808e951844d3ae5567f8b314333c: Deleting consensus metadata
I20260806 18:34:54.682789 23238 catalog_manager.cc:4841] TS 08f53e3bfff74ff4952e2cfe43750601 (127.18.125.131:43225): tablet 5d760af0344d40c9b603b7db4b4eccd5 (table test-table [id=4925b00b4baa4f92a12d92da0c5eca56]) successfully deleted
W20260806 18:34:54.683085 23239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:46591 (error 108)
I20260806 18:34:54.683158 23238 catalog_manager.cc:4841] TS 8ce2808e951844d3ae5567f8b314333c (127.18.125.130:45791): tablet 5d760af0344d40c9b603b7db4b4eccd5 (table test-table [id=4925b00b4baa4f92a12d92da0c5eca56]) successfully deleted
I20260806 18:34:54.683945 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 23459
I20260806 18:34:54.684191 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.683847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:46591 (local address 127.0.0.1:35238)
0806 18:34:54.683997 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:54.684072 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:46591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 18:34:54.684648 23239 catalog_manager.cc:4584] TS 7bfbab1fd3da4cd08631d4a7e58809c1 (127.18.125.129:46591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5d760af0344d40c9b603b7db4b4eccd5: Network error: Client connection negotiation failed: client connection to 127.18.125.129:46591: connect: Connection refused (error 111)
I20260806 18:34:54.691747 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 23601
I20260806 18:34:54.698482 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 23222
2026-08-06T18:34:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2542 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T18:34:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:54Z Disabled control of system clock
I20260806 18:34:54.731284 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:38977
--webserver_interface=127.18.125.190
--webserver_port=0
--builtin_ntp_servers=127.18.125.148:41407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:38977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:34:54.857445 23787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:54.857736 23787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:54.857776 23787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:54.861896 23787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:54.861966 23787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:54.861982 23787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:54.862000 23787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:54.862016 23787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:54.867000 23787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:38977
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:38977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:54.868106 23787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:54.869534 23787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:54.878227 23795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:54.878218 23793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:54.878424 23787 server_base.cc:1034] running on GCE node
W20260806 18:34:54.878198 23792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:54.879127 23787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:54.880311 23787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:54.881507 23787 hybrid_clock.cc:648] HybridClock initialized: now 1786041294881487 us; error 33 us; skew 500 ppm
I20260806 18:34:54.883493 23787 webserver.cc:460] Webserver started at http://127.18.125.190:38859/ using document root <none> and password file <none>
I20260806 18:34:54.884171 23787 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:54.884268 23787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:54.884516 23787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:54.886634 23787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "02aece8e8ca34e04a54885693f26f170"
format_stamp: "Formatted at 2026-08-06 18:34:54 on dist-test-slave-8w9v"
server_key: "0cf8082c569429f470a982cda23def79"
server_key_iv: "2c2cf4c32a70c8a00c4737813a1e93dc"
server_key_version: "encryptionkey@0"
I20260806 18:34:54.887213 23787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "02aece8e8ca34e04a54885693f26f170"
format_stamp: "Formatted at 2026-08-06 18:34:54 on dist-test-slave-8w9v"
server_key: "0cf8082c569429f470a982cda23def79"
server_key_iv: "2c2cf4c32a70c8a00c4737813a1e93dc"
server_key_version: "encryptionkey@0"
I20260806 18:34:54.891151 23787 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:54.894124 23801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:54.895506 23787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:34:54.895670 23787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "02aece8e8ca34e04a54885693f26f170"
format_stamp: "Formatted at 2026-08-06 18:34:54 on dist-test-slave-8w9v"
server_key: "0cf8082c569429f470a982cda23def79"
server_key_iv: "2c2cf4c32a70c8a00c4737813a1e93dc"
server_key_version: "encryptionkey@0"
I20260806 18:34:54.895802 23787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:54.901237 23787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:54.901919 23787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:54.902124 23787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:54.909255 23787 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:38977
I20260806 18:34:54.909271 23856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:38977 every 8 connection(s)
I20260806 18:34:54.910203 23787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:54.913509 23858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:54.918149 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 23787
I20260806 18:34:54.918278 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:54.918565 18934 external_mini_cluster.cc:1428] Setting key 26d222067cbe03de5a83a8e78817c553
I20260806 18:34:54.919873 23858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170: Bootstrap starting.
I20260806 18:34:54.922336 23858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:54.923380 23858 log.cc:824] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:54.924697 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:54.920266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34442 (local address 127.18.125.190:38977)
0806 18:34:54.920932 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:54.920948 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:54.920994 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:34:54.921659 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:54.921677 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:54.922078 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:54.922430 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:54.923254 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:54.923259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:54.923270 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:54.923280 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:54.923340 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:54.923355 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:54.923475 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:54.923587 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 18:34:54.923858 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":129,"threads_started":1}
I20260806 18:34:54.925522 23858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170: No bootstrap required, opened a new log
I20260806 18:34:54.929363 23858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02aece8e8ca34e04a54885693f26f170" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38977 } }
I20260806 18:34:54.929632 23858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:54.929720 23858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02aece8e8ca34e04a54885693f26f170, State: Initialized, Role: FOLLOWER
I20260806 18:34:54.930145 23858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02aece8e8ca34e04a54885693f26f170" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38977 } }
I20260806 18:34:54.930297 23858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:54.930390 23858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:54.930507 23858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:54.931557 23858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02aece8e8ca34e04a54885693f26f170" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38977 } }
I20260806 18:34:54.931905 23858 leader_election.cc:302] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02aece8e8ca34e04a54885693f26f170; no voters:
I20260806 18:34:54.932314 23858 leader_election.cc:288] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:54.932541 23863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:54.933012 23863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [term 1 LEADER]: Becoming Leader. State: Replica: 02aece8e8ca34e04a54885693f26f170, State: Running, Role: LEADER
I20260806 18:34:54.933313 23863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02aece8e8ca34e04a54885693f26f170" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38977 } }
I20260806 18:34:54.933701 23858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:54.934969 23864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02aece8e8ca34e04a54885693f26f170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02aece8e8ca34e04a54885693f26f170" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38977 } } }
I20260806 18:34:54.935096 23864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:54.935302 23865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02aece8e8ca34e04a54885693f26f170. Latest consensus state: current_term: 1 leader_uuid: "02aece8e8ca34e04a54885693f26f170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02aece8e8ca34e04a54885693f26f170" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 38977 } } }
I20260806 18:34:54.935477 23865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:54.935532 23872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:54.938470 23872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:54.943813 23872 catalog_manager.cc:1347] Generated new cluster ID: a279fffd04d54b138c60c35e41d8498f
I20260806 18:34:54.943900 23872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:54.956452 23872 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:54.956827 23872 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:54.964986 23872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170: Generated new TSK 0
I20260806 18:34:54.965736 23872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:54.972352 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:38977
--builtin_ntp_servers=127.18.125.148:41407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:34:55.093521 23882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:55.093829 23882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:55.093874 23882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:55.097975 23882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:55.098045 23882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:55.098107 23882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:34:55.103457 23882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:38977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:55.104628 23882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:55.106026 23882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:55.116246 23888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:55.116586 23882 server_base.cc:1034] running on GCE node
W20260806 18:34:55.116320 23889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:55.116246 23892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:55.117368 23882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:55.118309 23882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:55.119539 23882 hybrid_clock.cc:648] HybridClock initialized: now 1786041295119486 us; error 68 us; skew 500 ppm
I20260806 18:34:55.122014 23882 webserver.cc:460] Webserver started at http://127.18.125.129:37363/ using document root <none> and password file <none>
I20260806 18:34:55.122857 23882 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:55.122968 23882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:55.123245 23882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:55.125597 23882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "3175f5a78f304ef9a46be6ae5559e8ed"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "f0b5f54cc1d100393b474d9ac9a2fa43"
server_key_iv: "a623df4f2cd862f82a16c5e70ad58145"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.126216 23882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "3175f5a78f304ef9a46be6ae5559e8ed"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "f0b5f54cc1d100393b474d9ac9a2fa43"
server_key_iv: "a623df4f2cd862f82a16c5e70ad58145"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.130355 23882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:55.133271 23898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:55.134711 23882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:55.134853 23882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "3175f5a78f304ef9a46be6ae5559e8ed"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "f0b5f54cc1d100393b474d9ac9a2fa43"
server_key_iv: "a623df4f2cd862f82a16c5e70ad58145"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.134980 23882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:55.139963 23882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:55.140725 23882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:55.140893 23882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:55.141427 23882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:55.142383 23882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:55.142434 23882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:55.142474 23882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:55.142496 23882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:55.152005 23882 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:38587
I20260806 18:34:55.152026 24017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:38587 every 8 connection(s)
I20260806 18:34:55.153038 23882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:55.159875 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 23882
I20260806 18:34:55.160046 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:55.160370 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.155137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:59823 (local address 127.18.125.190:38977)
0806 18:34:55.155306 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.155311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.156608 (+ 1297us) server_negotiation.cc:407] Connection header received
0806 18:34:55.158178 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.158183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.158253 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.158385 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.159668 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:55.159673 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:55.159678 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:55.159705 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:55.159742 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:55.159756 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:55.159902 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:55.160150 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 18:34:55.160213 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:34:55.160435 18934 external_mini_cluster.cc:1428] Setting key da9fdf66ebfb2a13116d67b0e388d069
I20260806 18:34:55.161403 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.155571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:38977 (local address 127.18.125.129:59823)
0806 18:34:55.156442 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.156487 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:55.157836 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:55.158570 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:55.158578 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:55.158670 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:55.159303 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:55.159305 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:55.159355 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:55.159430 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:55.159905 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:55.159915 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:55.159921 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:34:55.160172 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 18:34:55.160442 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":123,"threads_started":1}
I20260806 18:34:55.162635 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:38977
--builtin_ntp_servers=127.18.125.148:41407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 18:34:55.162739 24018 heartbeater.cc:347] Connected to a master server at 127.18.125.190:38977
I20260806 18:34:55.162985 24018 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:55.163512 24018 heartbeater.cc:511] Master 127.18.125.190:38977 requested a full tablet report, sending...
I20260806 18:34:55.165285 23819 ts_manager.cc:194] Registered new tserver with Master: 3175f5a78f304ef9a46be6ae5559e8ed (127.18.125.129:38587)
W20260806 18:34:55.283392 24022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:55.283666 24022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:55.283712 24022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:55.287818 24022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:55.287885 24022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:55.287945 24022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:34:55.292860 24022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:38977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:55.293979 24022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:55.295346 24022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:55.305593 24031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:55.305615 24028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:55.305615 24029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:55.305676 24022 server_base.cc:1034] running on GCE node
I20260806 18:34:55.306345 24022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:55.307065 24022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:55.308245 24022 hybrid_clock.cc:648] HybridClock initialized: now 1786041295308212 us; error 45 us; skew 500 ppm
I20260806 18:34:55.310319 24022 webserver.cc:460] Webserver started at http://127.18.125.130:44749/ using document root <none> and password file <none>
I20260806 18:34:55.310992 24022 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:55.311090 24022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:55.311347 24022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:55.313599 24022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "5539eff37f614df98e20ac0f4b8aaa89"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "77a2c7803a33d17a8c99ab8145a6c9d3"
server_key_iv: "d52268bfda261c28f1108289e39b459f"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.314200 24022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "5539eff37f614df98e20ac0f4b8aaa89"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "77a2c7803a33d17a8c99ab8145a6c9d3"
server_key_iv: "d52268bfda261c28f1108289e39b459f"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.318635 24022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 18:34:55.321762 24037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:55.323155 24022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:34:55.323311 24022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "5539eff37f614df98e20ac0f4b8aaa89"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "77a2c7803a33d17a8c99ab8145a6c9d3"
server_key_iv: "d52268bfda261c28f1108289e39b459f"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.323458 24022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:55.328605 24022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:55.329332 24022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:55.329559 24022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:55.330199 24022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:55.331244 24022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:55.331321 24022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:55.331408 24022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:55.331451 24022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:55.341270 24022 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:32813
I20260806 18:34:55.341369 24160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:32813 every 8 connection(s)
I20260806 18:34:55.342330 24022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:55.348672 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.344336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:47529 (local address 127.18.125.190:38977)
0806 18:34:55.344489 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.344493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.345534 (+ 1041us) server_negotiation.cc:407] Connection header received
0806 18:34:55.346872 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.346875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.346918 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.347004 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.348253 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:55.348257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:55.348258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:55.348271 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:55.348291 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:55.348300 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:55.348408 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:55.348514 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 18:34:55.348555 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:34:55.349550 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.344699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:38977 (local address 127.18.125.130:47529)
0806 18:34:55.345382 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.345424 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:55.346640 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:55.347180 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:55.347187 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:55.347279 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:55.347931 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:55.347933 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:55.347976 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:55.348048 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:55.348383 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:55.348390 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:55.348394 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:34:55.348534 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 18:34:55.348739 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":122,"threads_started":1}
I20260806 18:34:55.350713 24161 heartbeater.cc:347] Connected to a master server at 127.18.125.190:38977
I20260806 18:34:55.350930 24161 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:55.351374 24161 heartbeater.cc:511] Master 127.18.125.190:38977 requested a full tablet report, sending...
I20260806 18:34:55.351629 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 24022
I20260806 18:34:55.351748 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:55.352070 18934 external_mini_cluster.cc:1428] Setting key 5d88edaa1019fb50a6b381ab6f8ce3f9
I20260806 18:34:55.352824 23819 ts_manager.cc:194] Registered new tserver with Master: 5539eff37f614df98e20ac0f4b8aaa89 (127.18.125.130:32813)
I20260806 18:34:55.354413 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:38977
--builtin_ntp_servers=127.18.125.148:41407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:34:55.472957 24164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:55.473248 24164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:55.473315 24164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:55.477531 24164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:55.477593 24164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:55.477686 24164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:34:55.482813 24164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:38977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:55.484057 24164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:55.485579 24164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:55.496536 24170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:55.496567 24164 server_base.cc:1034] running on GCE node
W20260806 18:34:55.496568 24169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:55.496593 24172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:55.497340 24164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:55.498072 24164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:55.499274 24164 hybrid_clock.cc:648] HybridClock initialized: now 1786041295499255 us; error 32 us; skew 500 ppm
I20260806 18:34:55.501458 24164 webserver.cc:460] Webserver started at http://127.18.125.131:35273/ using document root <none> and password file <none>
I20260806 18:34:55.502084 24164 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:55.502177 24164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:55.502435 24164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:55.504751 24164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "8ca2d52db8cf402886c7ca2ff5d06b21"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "b1cbfc149bf6217a5145377737f3a056"
server_key_iv: "64d768a315015fe387005eea69080688"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.505393 24164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "8ca2d52db8cf402886c7ca2ff5d06b21"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "b1cbfc149bf6217a5145377737f3a056"
server_key_iv: "64d768a315015fe387005eea69080688"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.509604 24164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:34:55.512717 24178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:55.514160 24164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:55.514317 24164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "8ca2d52db8cf402886c7ca2ff5d06b21"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "b1cbfc149bf6217a5145377737f3a056"
server_key_iv: "64d768a315015fe387005eea69080688"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.514469 24164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:55.519538 24164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:55.520299 24164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:55.520495 24164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:55.521104 24164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:55.522146 24164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:55.522220 24164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:55.522290 24164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:55.522318 24164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:55.532063 24164 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:41967
I20260806 18:34:55.532086 24296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:41967 every 8 connection(s)
I20260806 18:34:55.533079 24164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:34:55.539724 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.535154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:58639 (local address 127.18.125.190:38977)
0806 18:34:55.535369 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.535374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.536575 (+ 1201us) server_negotiation.cc:407] Connection header received
0806 18:34:55.537954 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.537956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.537996 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.538077 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.539297 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:55.539300 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:55.539302 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:55.539314 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:55.539334 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:55.539341 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:55.539442 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:55.539566 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 18:34:55.539607 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 18:34:55.540817 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.535616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:38977 (local address 127.18.125.131:58639)
0806 18:34:55.536421 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.536467 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:55.537750 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:55.538256 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:55.538265 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:55.538356 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:55.539025 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:55.539027 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:55.539070 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:55.539142 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:55.539440 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:55.539447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:55.539452 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:34:55.539594 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 18:34:55.539925 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":132,"threads_started":1}
I20260806 18:34:55.542001 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 24164
I20260806 18:34:55.542167 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:34:55.542536 24297 heartbeater.cc:347] Connected to a master server at 127.18.125.190:38977
I20260806 18:34:55.542793 18934 external_mini_cluster.cc:1428] Setting key 9be1d63eb1dc0b507b6f1d5d1dd98a7c
I20260806 18:34:55.542842 24297 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:55.543359 24297 heartbeater.cc:511] Master 127.18.125.190:38977 requested a full tablet report, sending...
I20260806 18:34:55.544587 23819 ts_manager.cc:194] Registered new tserver with Master: 8ca2d52db8cf402886c7ca2ff5d06b21 (127.18.125.131:41967)
I20260806 18:34:55.557435 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:34:55.561282 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.559517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.18.125.190:38977)
0806 18:34:55.559707 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.559712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.559858 (+ 146us) server_negotiation.cc:407] Connection header received
0806 18:34:55.559956 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.559958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.559995 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.560061 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.560862 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:55.560865 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:55.560867 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:55.560878 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:55.560899 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:55.560909 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:55.560987 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:55.561066 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 18:34:55.561102 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 18:34:55.564627 23819 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:34:55.567934 23819 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:34:55.581029 24313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.578207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37458 (local address 127.18.125.130:32813)
0806 18:34:55.578721 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.578735 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.578758 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:34:55.578814 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.578821 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.578992 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.579580 (+ 588us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.580547 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:55.580551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:55.580559 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:55.580570 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:55.580604 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:55.580631 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:55.580730 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:55.580782 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 18:34:55.580863 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":159,"threads_started":1}
I20260806 18:34:55.581017 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.577242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:38587 (local address 127.0.0.1:43492)
0806 18:34:55.577743 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.577794 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:55.577924 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:55.579224 (+ 1300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:55.579229 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:55.579261 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:55.579992 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:55.579994 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:55.580016 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:55.580114 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:55.580740 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:55.580745 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:55.580751 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:34:55.580826 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:34:55.580859 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":108,"threads_started":1}
I20260806 18:34:55.581019 24314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.578374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:41967 (local address 127.0.0.1:50084)
0806 18:34:55.578619 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.578632 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:55.578700 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:55.579750 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:55.579753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:55.579767 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:55.580231 (+ 464us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:55.580233 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:55.580243 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:55.580262 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:55.580581 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:55.580591 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:55.580596 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:34:55.580706 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 18:34:55.580777 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":94,"threads_started":1}
I20260806 18:34:55.581175 24315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.578917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50084 (local address 127.18.125.131:41967)
0806 18:34:55.579372 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.579386 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.579409 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:34:55.579465 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.579472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.579623 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.579763 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.580402 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:55.580409 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:55.580420 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:55.580430 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:55.580474 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:55.580493 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:55.580611 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:55.580939 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 18:34:55.581011 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":146,"threads_started":1}
I20260806 18:34:55.581247 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.577979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:32813 (local address 127.0.0.1:37458)
0806 18:34:55.578273 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.578287 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:55.578383 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:55.579176 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:55.579182 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:55.579260 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:55.580025 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:55.580028 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:55.580042 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:55.580110 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:55.580702 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:55.580704 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:55.580707 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:34:55.580767 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 18:34:55.580791 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":135,"threads_started":1}
I20260806 18:34:55.581902 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.577401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43492 (local address 127.18.125.129:38587)
0806 18:34:55.577802 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.577815 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.577889 (+ 74us) server_negotiation.cc:407] Connection header received
0806 18:34:55.578057 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.578067 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.578283 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.578463 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.580448 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:55.580455 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:55.580467 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:55.580482 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:55.580529 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:55.580569 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:55.580690 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:55.581631 (+ 941us) server_negotiation.cc:299] Negotiation successful
0806 18:34:55.581708 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":148,"threads_started":1}
I20260806 18:34:55.584656 24227 tablet_service.cc:1467] Processing CreateTablet for tablet c6ebdb9fee334adaa35ee3010bd488bf (DEFAULT_TABLE table=test-table [id=ab826b954e24413aa4d411eb6c1841d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:55.584656 24090 tablet_service.cc:1467] Processing CreateTablet for tablet c6ebdb9fee334adaa35ee3010bd488bf (DEFAULT_TABLE table=test-table [id=ab826b954e24413aa4d411eb6c1841d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:55.585883 24090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ebdb9fee334adaa35ee3010bd488bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:55.585883 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ebdb9fee334adaa35ee3010bd488bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:55.586148 23947 tablet_service.cc:1467] Processing CreateTablet for tablet c6ebdb9fee334adaa35ee3010bd488bf (DEFAULT_TABLE table=test-table [id=ab826b954e24413aa4d411eb6c1841d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:34:55.586934 23947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ebdb9fee334adaa35ee3010bd488bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:55.593299 24316 tablet_bootstrap.cc:492] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed: Bootstrap starting.
I20260806 18:34:55.594472 24317 tablet_bootstrap.cc:492] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89: Bootstrap starting.
I20260806 18:34:55.595376 24318 tablet_bootstrap.cc:492] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21: Bootstrap starting.
I20260806 18:34:55.596524 24317 tablet_bootstrap.cc:654] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:55.596546 24316 tablet_bootstrap.cc:654] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:55.597289 24317 log.cc:824] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:55.597396 24316 log.cc:824] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:55.597548 24318 tablet_bootstrap.cc:654] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:55.598327 24318 log.cc:824] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:55.599537 24316 tablet_bootstrap.cc:492] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed: No bootstrap required, opened a new log
I20260806 18:34:55.599550 24317 tablet_bootstrap.cc:492] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89: No bootstrap required, opened a new log
I20260806 18:34:55.599748 24316 ts_tablet_manager.cc:1397] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:34:55.599745 24317 ts_tablet_manager.cc:1397] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:34:55.600535 24318 tablet_bootstrap.cc:492] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21: No bootstrap required, opened a new log
I20260806 18:34:55.600793 24318 ts_tablet_manager.cc:1397] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:34:55.603593 24316 raft_consensus.cc:348] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38587 } } peers { permanent_uuid: "5539eff37f614df98e20ac0f4b8aaa89" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 32813 } } peers { permanent_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41967 } }
I20260806 18:34:55.603863 24316 raft_consensus.cc:374] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:55.603946 24316 raft_consensus.cc:729] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3175f5a78f304ef9a46be6ae5559e8ed, State: Initialized, Role: FOLLOWER
I20260806 18:34:55.604503 24316 consensus_queue.cc:260] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38587 } } peers { permanent_uuid: "5539eff37f614df98e20ac0f4b8aaa89" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 32813 } } peers { permanent_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41967 } }
I20260806 18:34:55.604601 24317 raft_consensus.cc:348] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38587 } } peers { permanent_uuid: "5539eff37f614df98e20ac0f4b8aaa89" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 32813 } } peers { permanent_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41967 } }
I20260806 18:34:55.604938 24317 raft_consensus.cc:374] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:55.605036 24317 raft_consensus.cc:729] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5539eff37f614df98e20ac0f4b8aaa89, State: Initialized, Role: FOLLOWER
I20260806 18:34:55.605216 24018 heartbeater.cc:503] Master 127.18.125.190:38977 was elected leader, sending a full tablet report...
I20260806 18:34:55.605595 24316 ts_tablet_manager.cc:1428] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:34:55.605473 24318 raft_consensus.cc:348] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38587 } } peers { permanent_uuid: "5539eff37f614df98e20ac0f4b8aaa89" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 32813 } } peers { permanent_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41967 } }
I20260806 18:34:55.605592 24317 consensus_queue.cc:260] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38587 } } peers { permanent_uuid: "5539eff37f614df98e20ac0f4b8aaa89" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 32813 } } peers { permanent_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41967 } }
I20260806 18:34:55.605767 24318 raft_consensus.cc:374] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:55.605844 24318 raft_consensus.cc:729] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ca2d52db8cf402886c7ca2ff5d06b21, State: Initialized, Role: FOLLOWER
I20260806 18:34:55.606333 24318 consensus_queue.cc:260] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38587 } } peers { permanent_uuid: "5539eff37f614df98e20ac0f4b8aaa89" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 32813 } } peers { permanent_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41967 } }
I20260806 18:34:55.606458 24161 heartbeater.cc:503] Master 127.18.125.190:38977 was elected leader, sending a full tablet report...
I20260806 18:34:55.606979 24317 ts_tablet_manager.cc:1428] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 18:34:55.607280 24318 ts_tablet_manager.cc:1428] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 18:34:55.607529 24297 heartbeater.cc:503] Master 127.18.125.190:38977 was elected leader, sending a full tablet report...
W20260806 18:34:55.654354 24019 tablet.cc:2369] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:34:55.681123 24322 raft_consensus.cc:482] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:34:55.681429 24322 raft_consensus.cc:504] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38587 } } peers { permanent_uuid: "5539eff37f614df98e20ac0f4b8aaa89" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 32813 } } peers { permanent_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41967 } }
I20260806 18:34:55.682765 24322 leader_election.cc:288] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5539eff37f614df98e20ac0f4b8aaa89 (127.18.125.130:32813), 8ca2d52db8cf402886c7ca2ff5d06b21 (127.18.125.131:41967)
I20260806 18:34:55.685197 24313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.683210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:37859 (local address 127.18.125.130:32813)
0806 18:34:55.683390 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.683394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.683497 (+ 103us) server_negotiation.cc:407] Connection header received
0806 18:34:55.683643 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.683646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.683694 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.683810 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.684749 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:55.684753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:55.684756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:55.684769 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:55.684793 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:55.684804 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:55.684915 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:55.685012 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 18:34:55.685056 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 18:34:55.685197 24315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.683727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:45129 (local address 127.18.125.131:41967)
0806 18:34:55.683900 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.683906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.683924 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:34:55.683981 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.683985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.684030 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.684169 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.684749 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:55.684753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:55.684756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:55.684769 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:55.684793 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:55.684804 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:55.684908 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:55.685012 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 18:34:55.685056 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 18:34:55.685226 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.683056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:32813 (local address 127.18.125.129:37859)
0806 18:34:55.683381 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.683401 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:55.683519 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:55.683871 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:55.683877 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:55.683898 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:55.684508 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:55.684510 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:55.684521 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:55.684553 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:55.684904 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:55.684908 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:55.684911 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:34:55.684995 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:34:55.685061 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":19,"thread_start_us":133,"threads_started":1}
I20260806 18:34:55.685243 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.683522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:41967 (local address 127.18.125.129:45129)
0806 18:34:55.683801 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.683817 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:55.683899 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:55.684191 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:55.684193 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:55.684221 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:55.684566 (+ 345us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:34:55.684568 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:55.684576 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:34:55.684600 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:55.684904 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:55.684908 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:55.684911 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:34:55.684995 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:34:55.685050 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
I20260806 18:34:55.686079 24112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ebdb9fee334adaa35ee3010bd488bf" candidate_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5539eff37f614df98e20ac0f4b8aaa89" is_pre_election: true
I20260806 18:34:55.686084 24248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ebdb9fee334adaa35ee3010bd488bf" candidate_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" is_pre_election: true
I20260806 18:34:55.686403 24112 raft_consensus.cc:2454] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3175f5a78f304ef9a46be6ae5559e8ed in term 0.
I20260806 18:34:55.686481 24248 raft_consensus.cc:2454] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3175f5a78f304ef9a46be6ae5559e8ed in term 0.
I20260806 18:34:55.686914 23902 leader_election.cc:302] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3175f5a78f304ef9a46be6ae5559e8ed, 5539eff37f614df98e20ac0f4b8aaa89; no voters:
I20260806 18:34:55.687232 24322 raft_consensus.cc:2781] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:34:55.687343 24322 raft_consensus.cc:482] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:34:55.687378 24322 raft_consensus.cc:3037] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:55.688645 24322 raft_consensus.cc:504] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38587 } } peers { permanent_uuid: "5539eff37f614df98e20ac0f4b8aaa89" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 32813 } } peers { permanent_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41967 } }
I20260806 18:34:55.689023 24322 leader_election.cc:288] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [CANDIDATE]: Term 1 election: Requested vote from peers 5539eff37f614df98e20ac0f4b8aaa89 (127.18.125.130:32813), 8ca2d52db8cf402886c7ca2ff5d06b21 (127.18.125.131:41967)
I20260806 18:34:55.689496 24112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ebdb9fee334adaa35ee3010bd488bf" candidate_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5539eff37f614df98e20ac0f4b8aaa89"
I20260806 18:34:55.689584 24248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ebdb9fee334adaa35ee3010bd488bf" candidate_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21"
I20260806 18:34:55.689666 24112 raft_consensus.cc:3037] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:55.689790 24248 raft_consensus.cc:3037] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:55.690897 24112 raft_consensus.cc:2454] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3175f5a78f304ef9a46be6ae5559e8ed in term 1.
I20260806 18:34:55.690907 24248 raft_consensus.cc:2454] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3175f5a78f304ef9a46be6ae5559e8ed in term 1.
I20260806 18:34:55.691227 23902 leader_election.cc:302] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3175f5a78f304ef9a46be6ae5559e8ed, 8ca2d52db8cf402886c7ca2ff5d06b21; no voters:
I20260806 18:34:55.691419 24322 raft_consensus.cc:2781] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:55.691697 24322 raft_consensus.cc:686] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 1 LEADER]: Becoming Leader. State: Replica: 3175f5a78f304ef9a46be6ae5559e8ed, State: Running, Role: LEADER
I20260806 18:34:55.692080 24322 consensus_queue.cc:237] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38587 } } peers { permanent_uuid: "5539eff37f614df98e20ac0f4b8aaa89" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 32813 } } peers { permanent_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41967 } }
I20260806 18:34:55.695776 23819 catalog_manager.cc:5495] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 3175f5a78f304ef9a46be6ae5559e8ed (127.18.125.129). New cstate: current_term: 1 leader_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3175f5a78f304ef9a46be6ae5559e8ed" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5539eff37f614df98e20ac0f4b8aaa89" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 32813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41967 } health_report { overall_health: UNKNOWN } } }
I20260806 18:34:55.706821 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.705204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43504 (local address 127.18.125.129:38587)
0806 18:34:55.705359 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:55.705364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:55.705379 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:34:55.705565 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:55.705573 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:55.705640 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:55.705733 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:55.706380 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:55.706383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:55.706386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:55.706400 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:55.706424 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:55.706440 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:55.706539 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:55.706623 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 18:34:55.706666 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:34:55.713260 24248 raft_consensus.cc:1261] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21 [term 1 FOLLOWER]: Refusing update from remote peer 3175f5a78f304ef9a46be6ae5559e8ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:55.713295 24112 raft_consensus.cc:1261] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89 [term 1 FOLLOWER]: Refusing update from remote peer 3175f5a78f304ef9a46be6ae5559e8ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:34:55.714155 24327 consensus_queue.cc:1035] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ca2d52db8cf402886c7ca2ff5d06b21" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:55.714416 24322 consensus_queue.cc:1035] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "5539eff37f614df98e20ac0f4b8aaa89" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 32813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:34:55.724488 24330 mvcc.cc:205] Tried to move back new op lower bound from 7315625147232714752 to 7315625147158011904. Current Snapshot: MvccSnapshot[applied={T|T < 7315625147232714752 or (T in {7315625147232714752})}]
I20260806 18:34:55.725291 24333 mvcc.cc:205] Tried to move back new op lower bound from 7315625147232714752 to 7315625147158011904. Current Snapshot: MvccSnapshot[applied={T|T < 7315625147232714752}]
I20260806 18:34:55.731976 23819 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:55.732193 23819 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:34:55.735054 23819 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02aece8e8ca34e04a54885693f26f170: Sending DeleteTablet for 3 replicas of tablet c6ebdb9fee334adaa35ee3010bd488bf
I20260806 18:34:55.736181 24090 tablet_service.cc:1514] Processing DeleteTablet for tablet c6ebdb9fee334adaa35ee3010bd488bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:55 UTC) from {username='slave'} at 127.0.0.1:37458
I20260806 18:34:55.736214 23947 tablet_service.cc:1514] Processing DeleteTablet for tablet c6ebdb9fee334adaa35ee3010bd488bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:55 UTC) from {username='slave'} at 127.0.0.1:43492
I20260806 18:34:55.736214 24227 tablet_service.cc:1514] Processing DeleteTablet for tablet c6ebdb9fee334adaa35ee3010bd488bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:34:55 UTC) from {username='slave'} at 127.0.0.1:50084
I20260806 18:34:55.736961 24342 tablet_replica.cc:330] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed: stopping tablet replica
I20260806 18:34:55.737337 24342 raft_consensus.cc:2229] T c6ebdb9fee334adaa35ee3010bd488bf P 3175f5a78f304ef9a46be6ae5559e8ed [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:34:55.737524 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 23882
I20260806 18:34:55.739351 24340 tablet_replica.cc:330] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89: stopping tablet replica
I20260806 18:34:55.739729 24340 raft_consensus.cc:2229] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:55.740147 24340 raft_consensus.cc:2258] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:55.740692 24341 tablet_replica.cc:330] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21: stopping tablet replica
I20260806 18:34:55.741024 24341 raft_consensus.cc:2229] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:34:55.741353 24341 raft_consensus.cc:2258] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:34:55.742156 24340 ts_tablet_manager.cc:1905] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:55.742733 24341 ts_tablet_manager.cc:1905] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:34:55.745302 24340 ts_tablet_manager.cc:1918] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:55.745472 24340 log.cc:1196] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/c6ebdb9fee334adaa35ee3010bd488bf
I20260806 18:34:55.745654 24341 ts_tablet_manager.cc:1918] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:34:55.745780 24341 log.cc:1196] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/c6ebdb9fee334adaa35ee3010bd488bf
I20260806 18:34:55.745855 24340 ts_tablet_manager.cc:1939] T c6ebdb9fee334adaa35ee3010bd488bf P 5539eff37f614df98e20ac0f4b8aaa89: Deleting consensus metadata
I20260806 18:34:55.746135 24341 ts_tablet_manager.cc:1939] T c6ebdb9fee334adaa35ee3010bd488bf P 8ca2d52db8cf402886c7ca2ff5d06b21: Deleting consensus metadata
I20260806 18:34:55.747314 23805 catalog_manager.cc:4841] TS 8ca2d52db8cf402886c7ca2ff5d06b21 (127.18.125.131:41967): tablet c6ebdb9fee334adaa35ee3010bd488bf (table test-table [id=ab826b954e24413aa4d411eb6c1841d0]) successfully deleted
I20260806 18:34:55.747550 23805 catalog_manager.cc:4841] TS 5539eff37f614df98e20ac0f4b8aaa89 (127.18.125.130:32813): tablet c6ebdb9fee334adaa35ee3010bd488bf (table test-table [id=ab826b954e24413aa4d411eb6c1841d0]) successfully deleted
W20260806 18:34:55.747800 23805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:38587 (error 108)
I20260806 18:34:55.747866 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 24022
I20260806 18:34:55.750881 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:55.750468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:38587 (local address 127.0.0.1:43512)
0806 18:34:55.750669 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:55.750763 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 18:34:55.751125 23805 catalog_manager.cc:4584] TS 3175f5a78f304ef9a46be6ae5559e8ed (127.18.125.129:38587): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6ebdb9fee334adaa35ee3010bd488bf: Network error: Client connection negotiation failed: client connection to 127.18.125.129:38587: connect: Connection refused (error 111)
I20260806 18:34:55.758872 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 24164
I20260806 18:34:55.770876 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 23787
2026-08-06T18:34:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1074 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 18:34:55.796963 18934 mini_oidc.cc:152] Starting JWKS server
I20260806 18:34:55.798584 18934 webserver.cc:460] Webserver started at https://127.0.0.1:45707/ using document root <none> and password file <none>
I20260806 18:34:55.798818 18934 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:34:55.799225 18934 webserver.cc:460] Webserver started at http://127.0.0.1:38275/ using document root <none> and password file <none>
2026-08-06T18:34:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:55Z Disabled control of system clock
I20260806 18:34:55.810300 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:37101
--webserver_interface=127.18.125.190
--webserver_port=0
--builtin_ntp_servers=127.18.125.148:41853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:37101
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45707/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/testchainca.cert with env {}
W20260806 18:34:55.928704 24350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:55.929051 24350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:55.929126 24350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:55.933337 24350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:55.933452 24350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:55.933483 24350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:55.933523 24350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:55.933580 24350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:34:55.933622 24350 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:34:55.933655 24350 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45707/jwks.json
I20260806 18:34:55.939393 24350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:37101
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:37101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45707/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:55.940650 24350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:55.941718 24350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:55.950560 24355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:55.950565 24358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:55.950560 24356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:55.950852 24350 server_base.cc:1034] running on GCE node
I20260806 18:34:55.951524 24350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:55.952772 24350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:55.953990 24350 hybrid_clock.cc:648] HybridClock initialized: now 1786041295953958 us; error 47 us; skew 500 ppm
I20260806 18:34:55.956348 24350 webserver.cc:460] Webserver started at http://127.18.125.190:41635/ using document root <none> and password file <none>
I20260806 18:34:55.956967 24350 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:55.957067 24350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:55.957335 24350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:55.959618 24350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "f0a3997dd56c4682af1b6a93e5ca87e5"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "808d6c482aafd4a3272242421e0f8b86"
server_key_iv: "e26233fa745752f16965f8c58a1beb5d"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.960278 24350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "f0a3997dd56c4682af1b6a93e5ca87e5"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "808d6c482aafd4a3272242421e0f8b86"
server_key_iv: "e26233fa745752f16965f8c58a1beb5d"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.964267 24350 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:34:55.967264 24364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:55.968739 24350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:55.968899 24350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "f0a3997dd56c4682af1b6a93e5ca87e5"
format_stamp: "Formatted at 2026-08-06 18:34:55 on dist-test-slave-8w9v"
server_key: "808d6c482aafd4a3272242421e0f8b86"
server_key_iv: "e26233fa745752f16965f8c58a1beb5d"
server_key_version: "encryptionkey@0"
I20260806 18:34:55.969031 24350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:55.987663 24350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:55.988541 24350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:55.988751 24350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:55.997133 24350 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:37101
I20260806 18:34:55.997146 24423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:37101 every 8 connection(s)
I20260806 18:34:55.998129 24350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:56.002517 24424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:56.007287 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 24350
I20260806 18:34:56.007449 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:56.007722 18934 external_mini_cluster.cc:1428] Setting key aaa746620085fe890d0868683425a1ac
I20260806 18:34:56.009419 24424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5: Bootstrap starting.
I20260806 18:34:56.011945 24424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:56.012794 24424 log.cc:824] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:56.015239 24424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5: No bootstrap required, opened a new log
I20260806 18:34:56.018515 24427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:56.009315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45286 (local address 127.18.125.190:37101)
0806 18:34:56.010061 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:56.010077 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:56.010116 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:34:56.010793 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:56.010822 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:56.011242 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:56.011616 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:56.012217 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.013397 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.014740 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.015259 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.015490 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.016515 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:56.016521 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:56.016536 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:56.017033 (+ 497us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:56.017100 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:56.017123 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:56.017251 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:56.017416 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 18:34:56.017711 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":160,"threads_started":1}
I20260806 18:34:56.019446 24424 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a3997dd56c4682af1b6a93e5ca87e5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37101 } }
I20260806 18:34:56.019685 24424 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:56.019727 24424 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a3997dd56c4682af1b6a93e5ca87e5, State: Initialized, Role: FOLLOWER
I20260806 18:34:56.020272 24424 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a3997dd56c4682af1b6a93e5ca87e5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37101 } }
I20260806 18:34:56.020407 24424 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:56.020495 24424 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:56.020619 24424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:56.021878 24424 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a3997dd56c4682af1b6a93e5ca87e5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37101 } }
I20260806 18:34:56.022174 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:56.022284 24424 leader_election.cc:302] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0a3997dd56c4682af1b6a93e5ca87e5; no voters:
I20260806 18:34:56.022673 24424 leader_election.cc:288] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:56.022764 24430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:56.022981 24430 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [term 1 LEADER]: Becoming Leader. State: Replica: f0a3997dd56c4682af1b6a93e5ca87e5, State: Running, Role: LEADER
I20260806 18:34:56.023530 24430 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a3997dd56c4682af1b6a93e5ca87e5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37101 } }
I20260806 18:34:56.024425 24424 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:56.025688 24431 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0a3997dd56c4682af1b6a93e5ca87e5. Latest consensus state: current_term: 1 leader_uuid: "f0a3997dd56c4682af1b6a93e5ca87e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a3997dd56c4682af1b6a93e5ca87e5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37101 } } }
I20260806 18:34:56.025655 24432 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0a3997dd56c4682af1b6a93e5ca87e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a3997dd56c4682af1b6a93e5ca87e5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37101 } } }
I20260806 18:34:56.025918 24431 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:56.025940 24432 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:56.026383 24439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:56.030054 24439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:56.036566 24439 catalog_manager.cc:1347] Generated new cluster ID: f76eea36fe04481f8e3bef157145cb52
I20260806 18:34:56.036693 24439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:56.048648 24439 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:56.049734 24439 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:56.064802 24439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0a3997dd56c4682af1b6a93e5ca87e5: Generated new TSK 0
I20260806 18:34:56.065755 24439 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 18:34:56.104087 24456 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 18:34:56.104344 24456 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 18:34:56.104920 24427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:56.086234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45290 (local address 127.18.125.190:37101)
0806 18:34:56.086376 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:56.086380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:56.086554 (+ 174us) server_negotiation.cc:407] Connection header received
0806 18:34:56.086747 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:56.086751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:56.086811 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:56.086940 (+ 129us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:34:56.087242 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.088103 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.089002 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.089348 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.089455 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.089819 (+ 364us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:34:56.104435 (+ 14616us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 18:34:56.104747 (+ 312us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45290: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 18:34:56.105010 24427 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45290: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 18:34:56.106077 24455 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:56.086126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:37101 (local address 127.0.0.1:45290)
0806 18:34:56.086464 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:56.086478 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:56.086588 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:56.086997 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:56.087000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:56.087009 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:34:56.087122 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.087127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.088248 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.088251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.088877 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.088885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.089475 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.089478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.089548 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.089595 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:34:56.104702 (+ 15107us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 18:34:56.104918 (+ 216us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.125.190:37101: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":117,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.125.190:37101: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45707/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 18:34:56.106758 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 24350
2026-08-06T18:34:56Z chronyd exiting
I20260806 18:34:56.145412 18934 test_util.cc:175] -----------------------------------------------
I20260806 18:34:56.145530 18934 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041269054381-18934-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (350 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 18:34:56.146679 18934 mini_oidc.cc:152] Starting JWKS server
I20260806 18:34:56.147615 18934 webserver.cc:460] Webserver started at https://127.0.0.1:36589/ using document root <none> and password file <none>
I20260806 18:34:56.147751 18934 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:34:56.148021 18934 webserver.cc:460] Webserver started at http://127.0.0.1:40777/ using document root <none> and password file <none>
2026-08-06T18:34:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:56Z Disabled control of system clock
I20260806 18:34:56.157480 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:32797
--webserver_interface=127.18.125.190
--webserver_port=0
--builtin_ntp_servers=127.18.125.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:32797
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36589/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/testchainca.cert with env {}
W20260806 18:34:56.283340 24462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:56.283754 24462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:56.283850 24462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:56.288276 24462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:56.288416 24462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:56.288447 24462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:56.288479 24462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:56.288530 24462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:34:56.288570 24462 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:34:56.288620 24462 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36589/jwks.json
I20260806 18:34:56.294858 24462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:32797
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:32797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36589/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:56.296195 24462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:56.297362 24462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:56.306828 24468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:56.306799 24467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:56.307150 24470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:56.307288 24462 server_base.cc:1034] running on GCE node
I20260806 18:34:56.308004 24462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:56.309333 24462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:56.310564 24462 hybrid_clock.cc:648] HybridClock initialized: now 1786041296310519 us; error 61 us; skew 500 ppm
I20260806 18:34:56.313864 24462 webserver.cc:460] Webserver started at http://127.18.125.190:41475/ using document root <none> and password file <none>
I20260806 18:34:56.314622 24462 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:56.314733 24462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:56.314997 24462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:56.317390 24462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "2c062a5338784e3ba4d0b3133632dcbc"
format_stamp: "Formatted at 2026-08-06 18:34:56 on dist-test-slave-8w9v"
server_key: "477bde0e5aa3b3e7000324e70cef9ec2"
server_key_iv: "8203bf1a4097ff1315c7400c5c171ca0"
server_key_version: "encryptionkey@0"
I20260806 18:34:56.318034 24462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "2c062a5338784e3ba4d0b3133632dcbc"
format_stamp: "Formatted at 2026-08-06 18:34:56 on dist-test-slave-8w9v"
server_key: "477bde0e5aa3b3e7000324e70cef9ec2"
server_key_iv: "8203bf1a4097ff1315c7400c5c171ca0"
server_key_version: "encryptionkey@0"
I20260806 18:34:56.322446 24462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:56.325827 24477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:56.327551 24462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:34:56.327752 24462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "2c062a5338784e3ba4d0b3133632dcbc"
format_stamp: "Formatted at 2026-08-06 18:34:56 on dist-test-slave-8w9v"
server_key: "477bde0e5aa3b3e7000324e70cef9ec2"
server_key_iv: "8203bf1a4097ff1315c7400c5c171ca0"
server_key_version: "encryptionkey@0"
I20260806 18:34:56.327924 24462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:56.349088 24462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:56.350132 24462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:56.350407 24462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:56.359226 24462 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:32797
I20260806 18:34:56.359269 24532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:32797 every 8 connection(s)
I20260806 18:34:56.360360 24462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:56.363938 24533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:56.365085 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 24462
I20260806 18:34:56.365190 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:56.365528 18934 external_mini_cluster.cc:1428] Setting key 6d51f424708999cd2a290ecd26c5b4e8
I20260806 18:34:56.370337 24533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc: Bootstrap starting.
I20260806 18:34:56.372877 24533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:56.373767 24533 log.cc:824] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:56.375572 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:56.366969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.18.125.190:32797)
0806 18:34:56.367723 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:56.367748 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:56.367798 (+ 50us) server_negotiation.cc:407] Connection header received
0806 18:34:56.368566 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:56.368585 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:56.369016 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:56.369370 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:56.369826 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.370896 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.371846 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.372415 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.372624 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.373641 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:56.373647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:56.373660 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:56.374141 (+ 481us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:56.374235 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:56.374265 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:56.374374 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:56.374546 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 18:34:56.374834 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":147,"threads_started":1}
I20260806 18:34:56.376091 24533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc: No bootstrap required, opened a new log
I20260806 18:34:56.378798 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:56.380312 24533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c062a5338784e3ba4d0b3133632dcbc" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32797 } }
I20260806 18:34:56.380573 24533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:56.380662 24533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c062a5338784e3ba4d0b3133632dcbc, State: Initialized, Role: FOLLOWER
I20260806 18:34:56.381126 24533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c062a5338784e3ba4d0b3133632dcbc" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32797 } }
I20260806 18:34:56.381254 24533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:56.381304 24533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:56.381376 24533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:56.382318 24533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c062a5338784e3ba4d0b3133632dcbc" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32797 } }
I20260806 18:34:56.382622 24533 leader_election.cc:302] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c062a5338784e3ba4d0b3133632dcbc; no voters:
I20260806 18:34:56.382896 24533 leader_election.cc:288] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:56.383064 24538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:56.383323 24538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [term 1 LEADER]: Becoming Leader. State: Replica: 2c062a5338784e3ba4d0b3133632dcbc, State: Running, Role: LEADER
I20260806 18:34:56.383659 24538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c062a5338784e3ba4d0b3133632dcbc" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32797 } }
I20260806 18:34:56.384298 24533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:56.385804 24539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c062a5338784e3ba4d0b3133632dcbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c062a5338784e3ba4d0b3133632dcbc" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32797 } } }
I20260806 18:34:56.385939 24539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:56.386361 24550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:56.386325 24540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c062a5338784e3ba4d0b3133632dcbc. Latest consensus state: current_term: 1 leader_uuid: "2c062a5338784e3ba4d0b3133632dcbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c062a5338784e3ba4d0b3133632dcbc" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32797 } } }
I20260806 18:34:56.387074 24540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:56.389878 24550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:56.396327 24550 catalog_manager.cc:1347] Generated new cluster ID: 8f3c3862a7a34be6827cd418af877752
I20260806 18:34:56.396425 24550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:56.413434 24550 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:56.414760 24550 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:56.425104 24550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c062a5338784e3ba4d0b3133632dcbc: Generated new TSK 0
I20260806 18:34:56.425966 24550 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 18:34:56.458717 24566 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 18:34:56.458863 24566 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 18:34:56.459780 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:56.442696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60664 (local address 127.18.125.190:32797)
0806 18:34:56.442856 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:56.442861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:56.443029 (+ 168us) server_negotiation.cc:407] Connection header received
0806 18:34:56.443178 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:56.443181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:56.443246 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:56.443379 (+ 133us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:34:56.443599 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.444491 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.445437 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.445798 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.445917 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.446530 (+ 613us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:34:56.459273 (+ 12743us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36589/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 18:34:56.459614 (+ 341us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60664: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36589/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 18:34:56.459853 24536 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60664: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36589/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 18:34:56.459904 24565 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:56.442544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:32797 (local address 127.0.0.1:60664)
0806 18:34:56.442937 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:56.442951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:56.443054 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:56.443384 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:56.443386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:56.443392 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:34:56.443494 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.443498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.444648 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.444651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.445315 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.445324 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.446138 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.446147 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.446226 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.446254 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:34:56.459527 (+ 13273us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36589/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 18:34:56.459695 (+ 168us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.125.190:32797: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36589/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":125,"threads_started":1}
I20260806 18:34:56.460414 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 24462
2026-08-06T18:34:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (354 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 18:34:56.501616 18934 mini_oidc.cc:152] Starting JWKS server
I20260806 18:34:56.502528 18934 webserver.cc:460] Webserver started at https://127.0.0.1:39069/ using document root <none> and password file <none>
I20260806 18:34:56.502640 18934 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:34:56.502928 18934 webserver.cc:460] Webserver started at http://127.0.0.1:39319/ using document root <none> and password file <none>
2026-08-06T18:34:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:56Z Disabled control of system clock
I20260806 18:34:56.512861 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:42297
--webserver_interface=127.18.125.190
--webserver_port=0
--builtin_ntp_servers=127.18.125.148:41249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:42297
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39069/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 18:34:56.633112 24572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:56.633476 24572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:56.633562 24572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:56.638527 24572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:56.638626 24572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:56.638659 24572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:56.638706 24572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:56.638747 24572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:34:56.638787 24572 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:34:56.638823 24572 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39069/jwks.json
I20260806 18:34:56.643993 24572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:42297
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:42297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39069/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:56.645378 24572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:56.646944 24572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:56.656459 24580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:56.656510 24578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:56.656412 24577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:56.656831 24572 server_base.cc:1034] running on GCE node
I20260806 18:34:56.657522 24572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:56.658766 24572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:56.659941 24572 hybrid_clock.cc:648] HybridClock initialized: now 1786041296659905 us; error 41 us; skew 500 ppm
I20260806 18:34:56.662352 24572 webserver.cc:460] Webserver started at http://127.18.125.190:35139/ using document root <none> and password file <none>
I20260806 18:34:56.663151 24572 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:56.663259 24572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:56.663519 24572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:56.665930 24572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "28849d1a61b44e88b61a0e1756136595"
format_stamp: "Formatted at 2026-08-06 18:34:56 on dist-test-slave-8w9v"
server_key: "15f97c64dffe557ffd8e1735ce904d24"
server_key_iv: "6f7b4ac064f3415dfe0ec8f04f7e2a7d"
server_key_version: "encryptionkey@0"
I20260806 18:34:56.666579 24572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "28849d1a61b44e88b61a0e1756136595"
format_stamp: "Formatted at 2026-08-06 18:34:56 on dist-test-slave-8w9v"
server_key: "15f97c64dffe557ffd8e1735ce904d24"
server_key_iv: "6f7b4ac064f3415dfe0ec8f04f7e2a7d"
server_key_version: "encryptionkey@0"
I20260806 18:34:56.670732 24572 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 18:34:56.674067 24586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:56.675437 24572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:34:56.675604 24572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "28849d1a61b44e88b61a0e1756136595"
format_stamp: "Formatted at 2026-08-06 18:34:56 on dist-test-slave-8w9v"
server_key: "15f97c64dffe557ffd8e1735ce904d24"
server_key_iv: "6f7b4ac064f3415dfe0ec8f04f7e2a7d"
server_key_version: "encryptionkey@0"
I20260806 18:34:56.675755 24572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:56.691331 24572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:56.692282 24572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:56.692520 24572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:56.702543 24572 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:42297
I20260806 18:34:56.702546 24642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:42297 every 8 connection(s)
I20260806 18:34:56.703939 24572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:56.707580 24643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:56.709808 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 24572
I20260806 18:34:56.709949 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:56.710239 18934 external_mini_cluster.cc:1428] Setting key 3fd3564ef5d47f55d7a43d1fe4ba670e
I20260806 18:34:56.714411 24643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595: Bootstrap starting.
I20260806 18:34:56.716959 24643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:56.717938 24643 log.cc:824] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:56.720752 24643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595: No bootstrap required, opened a new log
I20260806 18:34:56.721467 24646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:56.711817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50960 (local address 127.18.125.190:42297)
0806 18:34:56.712590 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:56.712607 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:56.712643 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:34:56.713682 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:56.713707 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:56.714161 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:56.714576 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:56.715077 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.716218 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.717179 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.717909 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.718102 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.719231 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:56.719237 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:56.719256 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:56.719766 (+ 510us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:56.719844 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:56.719872 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:56.720000 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:56.720164 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 18:34:56.720452 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":137,"threads_started":1}
I20260806 18:34:56.724588 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:56.725764 24643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28849d1a61b44e88b61a0e1756136595" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42297 } }
I20260806 18:34:56.726022 24643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:56.726096 24643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28849d1a61b44e88b61a0e1756136595, State: Initialized, Role: FOLLOWER
I20260806 18:34:56.726605 24643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28849d1a61b44e88b61a0e1756136595" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42297 } }
I20260806 18:34:56.726744 24643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:56.726835 24643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:56.726941 24643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:56.728073 24643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28849d1a61b44e88b61a0e1756136595" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42297 } }
I20260806 18:34:56.728507 24643 leader_election.cc:302] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28849d1a61b44e88b61a0e1756136595; no voters:
I20260806 18:34:56.728868 24643 leader_election.cc:288] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:56.729045 24648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:56.729329 24648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [term 1 LEADER]: Becoming Leader. State: Replica: 28849d1a61b44e88b61a0e1756136595, State: Running, Role: LEADER
I20260806 18:34:56.729676 24648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28849d1a61b44e88b61a0e1756136595" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42297 } }
I20260806 18:34:56.730249 24643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:56.731612 24650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28849d1a61b44e88b61a0e1756136595. Latest consensus state: current_term: 1 leader_uuid: "28849d1a61b44e88b61a0e1756136595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28849d1a61b44e88b61a0e1756136595" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42297 } } }
I20260806 18:34:56.731791 24650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:56.731601 24649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28849d1a61b44e88b61a0e1756136595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28849d1a61b44e88b61a0e1756136595" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42297 } } }
I20260806 18:34:56.731976 24649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:56.733282 24657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:56.736445 24657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:56.742635 24657 catalog_manager.cc:1347] Generated new cluster ID: bc168d674d1d418d93d10220163bdcd6
I20260806 18:34:56.742728 24657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:56.766770 24657 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:56.768255 24657 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:56.777760 24657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28849d1a61b44e88b61a0e1756136595: Generated new TSK 0
I20260806 18:34:56.778827 24657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:56.798686 24646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:56.793127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50972 (local address 127.18.125.190:42297)
0806 18:34:56.793287 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:56.793294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:56.793463 (+ 169us) server_negotiation.cc:407] Connection header received
0806 18:34:56.793636 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:56.793639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:56.793692 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:56.793793 (+ 101us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:34:56.794068 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.794620 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.795938 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.796251 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.796338 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.796718 (+ 380us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:34:56.798275 (+ 1557us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39069/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 18:34:56.798524 (+ 249us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39069/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 18:34:56.798759 24646 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39069/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 18:34:56.800616 24673 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:56.793039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42297 (local address 127.0.0.1:50972)
0806 18:34:56.793360 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:56.793378 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:56.793506 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:56.793818 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:56.793821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:56.793830 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:34:56.793963 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.793970 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.794744 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.794748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.795781 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:56.795793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:56.796397 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:56.796402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:56.796501 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:56.796533 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:34:56.798950 (+ 2417us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39069/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 18:34:56.800438 (+ 1488us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.125.190:42297: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39069/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":1269,"thread_start_us":114,"threads_started":1}
I20260806 18:34:56.801164 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 24572
E20260806 18:34:56.810849 24674 webserver.cc:533] Webserver: sslize failed:
E20260806 18:34:56.810958 24674 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T18:34:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (340 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T18:34:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:56Z Disabled control of system clock
I20260806 18:34:56.851123 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:39027
--webserver_interface=127.18.125.190
--webserver_port=0
--builtin_ntp_servers=127.18.125.148:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:39027
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/testchainca.cert with env {}
W20260806 18:34:56.968662 24679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:56.968971 24679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:56.969013 24679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:56.973102 24679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:56.973171 24679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:56.973187 24679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:56.973205 24679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:56.973220 24679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:34:56.973237 24679 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:34:56.973250 24679 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 18:34:56.979842 24679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:39027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:39027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:56.981199 24679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:56.982604 24679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:56.992690 24686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:56.992744 24679 server_base.cc:1034] running on GCE node
W20260806 18:34:56.992686 24685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:56.992682 24688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:56.993640 24679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:56.994870 24679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:56.996107 24679 hybrid_clock.cc:648] HybridClock initialized: now 1786041296996084 us; error 33 us; skew 500 ppm
I20260806 18:34:56.998234 24679 webserver.cc:460] Webserver started at http://127.18.125.190:39225/ using document root <none> and password file <none>
I20260806 18:34:56.998875 24679 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:56.998980 24679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:56.999267 24679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:57.001668 24679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "afe7d4e3ad384bbca3c93808a550ac30"
format_stamp: "Formatted at 2026-08-06 18:34:56 on dist-test-slave-8w9v"
server_key: "a99e67bbea51c416557eac567ef8bd51"
server_key_iv: "237b1368c01e4f25f74998931133aa7d"
server_key_version: "encryptionkey@0"
I20260806 18:34:57.002339 24679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "afe7d4e3ad384bbca3c93808a550ac30"
format_stamp: "Formatted at 2026-08-06 18:34:56 on dist-test-slave-8w9v"
server_key: "a99e67bbea51c416557eac567ef8bd51"
server_key_iv: "237b1368c01e4f25f74998931133aa7d"
server_key_version: "encryptionkey@0"
I20260806 18:34:57.006501 24679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:34:57.009344 24696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:57.010699 24679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:57.010864 24679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "afe7d4e3ad384bbca3c93808a550ac30"
format_stamp: "Formatted at 2026-08-06 18:34:56 on dist-test-slave-8w9v"
server_key: "a99e67bbea51c416557eac567ef8bd51"
server_key_iv: "237b1368c01e4f25f74998931133aa7d"
server_key_version: "encryptionkey@0"
I20260806 18:34:57.011006 24679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:57.032069 24679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:57.032953 24679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:57.033190 24679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:57.041046 24679 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:39027
I20260806 18:34:57.041055 24751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:39027 every 8 connection(s)
I20260806 18:34:57.042131 24679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:57.045225 24752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:57.048043 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 24679
I20260806 18:34:57.048204 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:57.048507 18934 external_mini_cluster.cc:1428] Setting key 83b44d91c07bee3c7f54867c54d2977b
I20260806 18:34:57.051637 24752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30: Bootstrap starting.
I20260806 18:34:57.054231 24752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:57.055116 24752 log.cc:824] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:57.057686 24752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30: No bootstrap required, opened a new log
I20260806 18:34:57.058686 24755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:57.049951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46856 (local address 127.18.125.190:39027)
0806 18:34:57.050630 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:57.050648 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:57.050687 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:34:57.051366 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:57.051383 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:57.051849 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:57.052229 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:57.052605 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.053628 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.054853 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.055419 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.055600 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:57.056470 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:57.056478 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:57.056495 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:34:57.057008 (+ 513us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:57.057083 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:34:57.057112 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:57.057247 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:57.057444 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 18:34:57.057727 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":164,"threads_started":1}
I20260806 18:34:57.061929 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:34:57.062350 24752 raft_consensus.cc:348] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe7d4e3ad384bbca3c93808a550ac30" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39027 } }
I20260806 18:34:57.062590 24752 raft_consensus.cc:374] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:57.062668 24752 raft_consensus.cc:729] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afe7d4e3ad384bbca3c93808a550ac30, State: Initialized, Role: FOLLOWER
I20260806 18:34:57.063194 24752 consensus_queue.cc:260] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe7d4e3ad384bbca3c93808a550ac30" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39027 } }
I20260806 18:34:57.063337 24752 raft_consensus.cc:388] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:57.063432 24752 raft_consensus.cc:482] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:57.063553 24752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:57.064730 24752 raft_consensus.cc:504] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe7d4e3ad384bbca3c93808a550ac30" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39027 } }
I20260806 18:34:57.065109 24752 leader_election.cc:302] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afe7d4e3ad384bbca3c93808a550ac30; no voters:
I20260806 18:34:57.065478 24752 leader_election.cc:288] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:57.065613 24757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:57.065881 24757 raft_consensus.cc:686] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [term 1 LEADER]: Becoming Leader. State: Replica: afe7d4e3ad384bbca3c93808a550ac30, State: Running, Role: LEADER
I20260806 18:34:57.066239 24757 consensus_queue.cc:237] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe7d4e3ad384bbca3c93808a550ac30" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39027 } }
I20260806 18:34:57.067001 24752 sys_catalog.cc:564] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:57.068310 24758 sys_catalog.cc:455] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afe7d4e3ad384bbca3c93808a550ac30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe7d4e3ad384bbca3c93808a550ac30" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39027 } } }
I20260806 18:34:57.068323 24759 sys_catalog.cc:455] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afe7d4e3ad384bbca3c93808a550ac30. Latest consensus state: current_term: 1 leader_uuid: "afe7d4e3ad384bbca3c93808a550ac30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe7d4e3ad384bbca3c93808a550ac30" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39027 } } }
I20260806 18:34:57.068487 24759 sys_catalog.cc:458] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:57.068486 24758 sys_catalog.cc:458] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:57.068966 24766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:57.072434 24766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:57.078347 24766 catalog_manager.cc:1347] Generated new cluster ID: cbfbbeb0f0934ae2b9c86dae19a5aafd
I20260806 18:34:57.078444 24766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:57.087365 24766 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:57.088639 24766 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:57.092486 24766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afe7d4e3ad384bbca3c93808a550ac30: Generated new TSK 0
I20260806 18:34:57.093685 24766 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:34:57.567821 24755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:57.103432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.18.125.190:39027)
0806 18:34:57.103592 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:57.103598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:57.103731 (+ 133us) server_negotiation.cc:407] Connection header received
0806 18:34:57.103954 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:57.103957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:57.104011 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:57.104172 (+ 161us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:34:57.104446 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.105004 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.105934 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.106184 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.106327 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:57.106696 (+ 369us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:34:57.567289 (+460593us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 18:34:57.567685 (+ 396us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46862: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 18:34:57.567942 24755 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46862: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 18:34:57.568104 24782 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:34:57.103289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39027 (local address 127.0.0.1:46862)
0806 18:34:57.103623 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:57.103637 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:57.103782 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:57.104177 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:57.104181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:57.104189 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:34:57.104322 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:57.104329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.105147 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.105152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:57.105797 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:57.105805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:57.106356 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:57.106360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:57.106466 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:57.106490 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:34:57.567573 (+461083us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 18:34:57.567874 (+ 301us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.125.190:39027: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":127,"threads_started":1}
I20260806 18:34:57.568822 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 24679
2026-08-06T18:34:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (747 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (134 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (139 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:34:57 dist-test-slave-8w9v krb5kdc[24795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:34:57 dist-test-slave-8w9v krb5kdc[24795](info): set up 2 sockets
Aug 06 18:34:57 dist-test-slave-8w9v krb5kdc[24795](info): commencing operation
krb5kdc: starting...
W20260806 18:34:58.927090 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:34:59 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:34:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:34:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:34:59.063887 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:41591
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:41591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:59.182578 24809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:59.182910 24809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:59.182991 24809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:59.187093 24809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:34:59.187216 24809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:59.187252 24809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:59.187299 24809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:34:59.187331 24809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:34:59.193586 24809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:41591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:41591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:59.194808 24809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:59.195895 24809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:34:59.205081 24809 server_base.cc:1034] running on GCE node
W20260806 18:34:59.205081 24815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:59.205086 24814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:59.205349 24817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:59.205999 24809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:59.207264 24809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:59.208460 24809 hybrid_clock.cc:648] HybridClock initialized: now 1786041299208428 us; error 48 us; skew 500 ppm
Aug 06 18:34:59 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041299, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:59.217089 24809 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:34:59.218276 24809 webserver.cc:460] Webserver started at http://127.18.125.190:35165/ using document root <none> and password file <none>
I20260806 18:34:59.218964 24809 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:59.219074 24809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:59.219343 24809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:59.221983 24809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "580239b1afd04abb83ccab97e870d395"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "26c0acb9b0666d6dee0a385e921f7521"
server_key_iv: "007f214851ca86f5b929df4115319b0e"
server_key_version: "encryptionkey@0"
I20260806 18:34:59.222651 24809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "580239b1afd04abb83ccab97e870d395"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "26c0acb9b0666d6dee0a385e921f7521"
server_key_iv: "007f214851ca86f5b929df4115319b0e"
server_key_version: "encryptionkey@0"
I20260806 18:34:59.227464 24809 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:34:59.230840 24824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:59.232524 24809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 18:34:59.232939 24809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "580239b1afd04abb83ccab97e870d395"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "26c0acb9b0666d6dee0a385e921f7521"
server_key_iv: "007f214851ca86f5b929df4115319b0e"
server_key_version: "encryptionkey@0"
I20260806 18:34:59.233115 24809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:59.246728 24809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:59.250761 24809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:59.251025 24809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:59.260423 24809 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:41591
I20260806 18:34:59.260463 24881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:41591 every 8 connection(s)
I20260806 18:34:59.261544 24809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:34:59.262176 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 24809
I20260806 18:34:59.262326 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:34:59.262616 18934 external_mini_cluster.cc:1428] Setting key 0cea86939a4c4747c4201274b8355f0b
I20260806 18:34:59.265188 24882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:34:59.272491 24882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395: Bootstrap starting.
Aug 06 18:34:59 dist-test-slave-8w9v krb5kdc[24795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:59.275279 24882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395: Neither blocks nor log segments found. Creating new log.
I20260806 18:34:59.276188 24882 log.cc:824] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395: Log is configured to *not* fsync() on all Append() calls
I20260806 18:34:59.278805 24882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395: No bootstrap required, opened a new log
I20260806 18:34:59.279920 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:59.264267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.18.125.190:41591)
0806 18:34:59.265173 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:59.265197 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:59.265242 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:34:59.266423 (+ 1181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:59.266447 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:59.266818 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:59.267217 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:59.268181 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.269317 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.270274 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.270793 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.271006 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:59.273872 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:59.273878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:59.273900 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:59.273938 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:59.276621 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:59.277180 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:59.277186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:59.277191 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:59.277340 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:59.277716 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:59.277720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:59.277721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:59.278196 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:59.278396 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:59.278725 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 18:34:59.279090 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":147,"threads_started":1}
I20260806 18:34:59.282395 24882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580239b1afd04abb83ccab97e870d395" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41591 } }
I20260806 18:34:59.282717 24882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:34:59.282789 24882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580239b1afd04abb83ccab97e870d395, State: Initialized, Role: FOLLOWER
I20260806 18:34:59.283257 24882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580239b1afd04abb83ccab97e870d395" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41591 } }
I20260806 18:34:59.283422 24882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:34:59.283509 24882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:34:59.283634 24882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:34:59.284791 24882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580239b1afd04abb83ccab97e870d395" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41591 } }
I20260806 18:34:59.285162 24882 leader_election.cc:302] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 580239b1afd04abb83ccab97e870d395; no voters:
I20260806 18:34:59.285578 24882 leader_election.cc:288] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:34:59.285758 24887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:34:59.286082 24887 raft_consensus.cc:686] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [term 1 LEADER]: Becoming Leader. State: Replica: 580239b1afd04abb83ccab97e870d395, State: Running, Role: LEADER
I20260806 18:34:59.286497 24887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580239b1afd04abb83ccab97e870d395" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41591 } }
I20260806 18:34:59.287070 24882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:34:59.288450 24889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "580239b1afd04abb83ccab97e870d395" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580239b1afd04abb83ccab97e870d395" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41591 } } }
I20260806 18:34:59.288595 24889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:59.288821 24888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 580239b1afd04abb83ccab97e870d395. Latest consensus state: current_term: 1 leader_uuid: "580239b1afd04abb83ccab97e870d395" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580239b1afd04abb83ccab97e870d395" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41591 } } }
I20260806 18:34:59.289009 24888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395 [sys.catalog]: This master's current role is: LEADER
I20260806 18:34:59.289031 24896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:34:59.294234 24896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:34:59.300841 24896 catalog_manager.cc:1347] Generated new cluster ID: a413a13dd1a2496a8eb053d0f4d9be43
I20260806 18:34:59.300951 24896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:34:59.315841 24896 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:34:59.317008 24896 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:34:59.324823 24896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 580239b1afd04abb83ccab97e870d395: Generated new TSK 0
I20260806 18:34:59.325639 24896 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:34:59.359936 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:41591
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:59.487018 24908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:59.487370 24908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:59.487450 24908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:59.492817 24908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:59.492902 24908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:59.493170 24908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:34:59.499778 24908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:41591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:59.501153 24908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:59.502331 24908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:59.513180 24917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:59.513180 24915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:59.513204 24914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:59.513831 24908 server_base.cc:1034] running on GCE node
I20260806 18:34:59.514217 24908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:59.514838 24908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:59.516086 24908 hybrid_clock.cc:648] HybridClock initialized: now 1786041299516067 us; error 38 us; skew 500 ppm
Aug 06 18:34:59 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041299, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:59.519290 24908 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:34:59.520342 24908 webserver.cc:460] Webserver started at http://127.18.125.129:38383/ using document root <none> and password file <none>
I20260806 18:34:59.520968 24908 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:59.521128 24908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:59.521442 24908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:59.523664 24908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "5d069afbab4c4c5a9f274202d8ae3f46"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "4d33f6f12d9dabceb5c6e9476f8a3b13"
server_key_iv: "36fadd88b751029912db2c1574e4a930"
server_key_version: "encryptionkey@0"
I20260806 18:34:59.524461 24908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "5d069afbab4c4c5a9f274202d8ae3f46"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "4d33f6f12d9dabceb5c6e9476f8a3b13"
server_key_iv: "36fadd88b751029912db2c1574e4a930"
server_key_version: "encryptionkey@0"
I20260806 18:34:59.528709 24908 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:34:59.531255 24924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:59.532615 24908 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:34:59.532747 24908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "5d069afbab4c4c5a9f274202d8ae3f46"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "4d33f6f12d9dabceb5c6e9476f8a3b13"
server_key_iv: "36fadd88b751029912db2c1574e4a930"
server_key_version: "encryptionkey@0"
I20260806 18:34:59.532860 24908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:59.543030 24908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:59.546281 24908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:59.546509 24908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:59.547152 24908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:59.548105 24908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:59.548216 24908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:59.548285 24908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:59.548326 24908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:59.557863 24908 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:43217
I20260806 18:34:59.557915 25047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:43217 every 8 connection(s)
I20260806 18:34:59.558804 24908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:34:59.568301 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 24908
I20260806 18:34:59.568497 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:34:59.568797 18934 external_mini_cluster.cc:1428] Setting key 6719dcdb07b781e49fecc36d45a01139
Aug 06 18:34:59 dist-test-slave-8w9v krb5kdc[24795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041299, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:59.574527 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:59.560742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:33645 (local address 127.18.125.190:41591)
0806 18:34:59.560896 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:59.560900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:59.561855 (+ 955us) server_negotiation.cc:407] Connection header received
0806 18:34:59.563280 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:59.563284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:59.563333 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:59.563406 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:59.564716 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.565274 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.566483 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.566718 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.566804 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:59.569837 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:59.569840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:59.569842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:59.569867 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:59.571460 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:59.572694 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:59.572702 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:59.572706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:59.572817 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:59.573279 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:59.573283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:59.573285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:59.573491 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:59.573607 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:59.574163 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 18:34:59.574346 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:34:59.575449 25050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:59.561070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:41591 (local address 127.18.125.129:33645)
0806 18:34:59.561667 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:59.561717 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:59.563054 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:59.563579 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:59.563585 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:59.563966 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:59.564543 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:59.564558 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.565436 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.565439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:59.566342 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:59.566353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.566868 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.566871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:59.567022 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:59.567701 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:59.567732 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:59.569648 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:59.571598 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:59.571605 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:59.571623 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:59.572441 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:59.572957 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:59.572961 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:59.572964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:59.573138 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:59.573608 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:59.573619 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:59.573904 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 18:34:59.574105 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 18:34:59.574473 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":129,"threads_started":1}
I20260806 18:34:59.576917 25048 heartbeater.cc:347] Connected to a master server at 127.18.125.190:41591
I20260806 18:34:59.577194 25048 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:59.577741 25048 heartbeater.cc:511] Master 127.18.125.190:41591 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:34:59.579425 24843 ts_manager.cc:194] Registered new tserver with Master: 5d069afbab4c4c5a9f274202d8ae3f46 (127.18.125.129:43217)
I20260806 18:34:59.580758 24843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:33645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:34:59.612177 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:41591
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:59.727407 25053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:59.727692 25053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:59.727737 25053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:59.731721 25053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:59.731786 25053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:59.731844 25053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:34:59.737504 25053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:41591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:59.738668 25053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:59.739781 25053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:59.751484 25058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:59.751632 25061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:59.751739 25059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:59.752250 25053 server_base.cc:1034] running on GCE node
I20260806 18:34:59.752799 25053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:59.753870 25053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:59.755110 25053 hybrid_clock.cc:648] HybridClock initialized: now 1786041299755081 us; error 54 us; skew 500 ppm
Aug 06 18:34:59 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041299, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:59.758410 25053 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:34:59.759335 25053 webserver.cc:460] Webserver started at http://127.18.125.130:34991/ using document root <none> and password file <none>
I20260806 18:34:59.759945 25053 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:59.760039 25053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:59.760315 25053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:59.762665 25053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "baa389f4247e4a5ba148a658a2b8bed4"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "09e5bc344dd4ba3129e17f35db429cda"
server_key_iv: "c8ce2b9d90c0b296291d68692157cac4"
server_key_version: "encryptionkey@0"
I20260806 18:34:59.763304 25053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "baa389f4247e4a5ba148a658a2b8bed4"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "09e5bc344dd4ba3129e17f35db429cda"
server_key_iv: "c8ce2b9d90c0b296291d68692157cac4"
server_key_version: "encryptionkey@0"
I20260806 18:34:59.768011 25053 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:34:59.771174 25069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:59.772634 25053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:34:59.772802 25053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "baa389f4247e4a5ba148a658a2b8bed4"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "09e5bc344dd4ba3129e17f35db429cda"
server_key_iv: "c8ce2b9d90c0b296291d68692157cac4"
server_key_version: "encryptionkey@0"
I20260806 18:34:59.772948 25053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:34:59.784659 25053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:34:59.788611 25053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:34:59.788846 25053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:34:59.789525 25053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:34:59.790457 25053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:34:59.790531 25053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:59.790607 25053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:34:59.790658 25053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:34:59.800271 25053 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:34721
I20260806 18:34:59.800289 25188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:34721 every 8 connection(s)
I20260806 18:34:59.801211 25053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:34:59.809497 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 25053
I20260806 18:34:59.809664 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:34:59.809968 18934 external_mini_cluster.cc:1428] Setting key 23cf961e67fe901b03cb551ff168b6f0
Aug 06 18:34:59 dist-test-slave-8w9v krb5kdc[24795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041299, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:34:59.817018 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:59.803095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:41339 (local address 127.18.125.190:41591)
0806 18:34:59.803242 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:34:59.803245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:34:59.804105 (+ 860us) server_negotiation.cc:407] Connection header received
0806 18:34:59.806050 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:34:59.806053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:34:59.806119 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:34:59.806235 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:34:59.807491 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.808043 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.809262 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.809492 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.809612 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:59.812574 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:34:59.812577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:34:59.812579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:34:59.812604 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:34:59.814507 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:59.815158 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:59.815163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:59.815165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:59.815261 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:34:59.815651 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:34:59.815654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:34:59.815656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:34:59.815892 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:34:59.816029 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:34:59.816677 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 18:34:59.816831 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:34:59.817900 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:34:59.803499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:41591 (local address 127.18.125.130:41339)
0806 18:34:59.803952 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 18:34:59.803996 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:34:59.805823 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:34:59.806432 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:34:59.806439 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:34:59.806821 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:34:59.807323 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:59.807337 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.808225 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.808229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:59.809126 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:34:59.809136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:34:59.809660 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:34:59.809665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:34:59.809890 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:34:59.810629 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:34:59.810651 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:34:59.812394 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:34:59.814665 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:59.814677 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:59.814694 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:59.815005 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:59.815370 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:34:59.815374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:34:59.815376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:34:59.815535 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:34:59.816039 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:34:59.816049 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:34:59.816413 (+ 364us) client_negotiation.cc:769] Sending connection context
0806 18:34:59.816614 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 18:34:59.816987 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":107,"threads_started":1}
I20260806 18:34:59.819905 25189 heartbeater.cc:347] Connected to a master server at 127.18.125.190:41591
I20260806 18:34:59.820515 25189 heartbeater.cc:464] Registering TS with master...
I20260806 18:34:59.821466 25189 heartbeater.cc:511] Master 127.18.125.190:41591 requested a full tablet report, sending...
I20260806 18:34:59.823575 24843 ts_manager.cc:194] Registered new tserver with Master: baa389f4247e4a5ba148a658a2b8bed4 (127.18.125.130:34721)
I20260806 18:34:59.824632 24843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:41339
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:34:59.847548 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:41591
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:34:59.965890 25194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:34:59.966150 25194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:34:59.966195 25194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:34:59.970364 25194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:34:59.970422 25194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:34:59.970482 25194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:34:59.975698 25194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:41591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:34:59.976949 25194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:34:59.978060 25194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:34:59.987807 25199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:59.987862 25200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:34:59.988003 25203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:34:59.988381 25194 server_base.cc:1034] running on GCE node
I20260806 18:34:59.988804 25194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:34:59.989512 25194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:34:59.990716 25194 hybrid_clock.cc:648] HybridClock initialized: now 1786041299990699 us; error 29 us; skew 500 ppm
Aug 06 18:34:59 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041299, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:34:59.993635 25194 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:34:59.994589 25194 webserver.cc:460] Webserver started at http://127.18.125.131:41809/ using document root <none> and password file <none>
I20260806 18:34:59.995172 25194 fs_manager.cc:359] Metadata directory not provided
I20260806 18:34:59.995297 25194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:34:59.995523 25194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:34:59.998111 25194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "012c5b94341b47348cb91924d27b8d20"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "c596600e2c26bbb27508020926cb7b7c"
server_key_iv: "f524ba88c568f6f4100d260cc0aa5e74"
server_key_version: "encryptionkey@0"
I20260806 18:34:59.998807 25194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "012c5b94341b47348cb91924d27b8d20"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "c596600e2c26bbb27508020926cb7b7c"
server_key_iv: "f524ba88c568f6f4100d260cc0aa5e74"
server_key_version: "encryptionkey@0"
I20260806 18:35:00.003424 25194 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:35:00.006166 25211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:00.007534 25194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 18:35:00.007701 25194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "012c5b94341b47348cb91924d27b8d20"
format_stamp: "Formatted at 2026-08-06 18:34:59 on dist-test-slave-8w9v"
server_key: "c596600e2c26bbb27508020926cb7b7c"
server_key_iv: "f524ba88c568f6f4100d260cc0aa5e74"
server_key_version: "encryptionkey@0"
I20260806 18:35:00.007833 25194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:00.021762 25194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:00.025305 25194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:00.025563 25194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:00.026194 25194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:00.027067 25194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:00.027140 25194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:00.027213 25194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:00.027266 25194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:00.036974 25194 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:35479
I20260806 18:35:00.037125 25330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:35479 every 8 connection(s)
I20260806 18:35:00.038093 25194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:00.045086 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 25194
I20260806 18:35:00.045188 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:35:00.045493 18934 external_mini_cluster.cc:1428] Setting key efbc4a24060c91985f2228230ce15156
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[24795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041299, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:35:00.052806 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:00.040036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:57503 (local address 127.18.125.190:41591)
0806 18:35:00.040238 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:00.040242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:00.041074 (+ 832us) server_negotiation.cc:407] Connection header received
0806 18:35:00.042466 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:00.042470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:00.042531 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:00.042641 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:00.043811 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.044408 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.045677 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.045911 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.046029 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:00.048750 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:00.048758 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:00.048761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:00.048795 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:00.050201 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:00.050792 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:00.050796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:00.050798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:00.050890 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:00.051512 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:00.051517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:00.051519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:00.051742 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:00.051865 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:00.052482 (+ 617us) server_negotiation.cc:299] Negotiation successful
0806 18:35:00.052623 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:35:00.053658 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:00.040456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:41591 (local address 127.18.125.131:57503)
0806 18:35:00.040921 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:00.040963 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:00.042251 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:00.042821 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:00.042827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:00.043179 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:00.043656 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:00.043669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.044540 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.044543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:00.045518 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:00.045550 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.046073 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.046078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:00.046307 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:00.046993 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:00.047011 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:00.048565 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:00.050349 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:00.050381 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:00.050405 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:00.050676 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:00.050992 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:00.050996 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:00.050998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:00.051379 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:00.051880 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:00.051889 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:00.052211 (+ 322us) client_negotiation.cc:769] Sending connection context
0806 18:35:00.052439 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 18:35:00.052796 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":114,"threads_started":1}
I20260806 18:35:00.054955 25331 heartbeater.cc:347] Connected to a master server at 127.18.125.190:41591
I20260806 18:35:00.055190 25331 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:00.055673 25331 heartbeater.cc:511] Master 127.18.125.190:41591 requested a full tablet report, sending...
I20260806 18:35:00.056783 24843 ts_manager.cc:194] Registered new tserver with Master: 012c5b94341b47348cb91924d27b8d20 (127.18.125.131:35479)
I20260806 18:35:00.057412 24843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:57503
I20260806 18:35:00.060096 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/test-admin.keytab.
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:00.078019 18934 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 18:35:00.081243 18934 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/krb5cc' of type FILE
W20260806 18:35:00.081343 25339 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:00.079515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42954 (local address 127.18.125.129:43217)
0806 18:35:00.079825 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:00.079832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:00.079857 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:35:00.079919 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:00.079924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:00.080109 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:00.080307 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:00.081060 (+ 753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42954: BlockingRecv error: recv got EOF from 127.0.0.1:42954 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":106,"threads_started":1}
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:00.082675 18934 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:00.083801 18934 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[24795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:35:00.095444 25339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:00.085306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.18.125.129:43217)
0806 18:35:00.085500 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:00.085504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:00.085522 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:35:00.085679 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:00.085683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:00.085734 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:00.085820 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:00.086473 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.087376 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.088398 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.088823 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.088916 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:00.091498 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:00.091511 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:00.091527 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:00.091555 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:00.093398 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:00.093913 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:00.093919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:00.093920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:00.094036 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:00.094311 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:00.094314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:00.094315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:00.094601 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:00.094764 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:00.095038 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 18:35:00.095242 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260806 18:35:00.099642 25339 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:00.098509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.18.125.129:43217)
0806 18:35:00.098603 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:00.098607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:00.098770 (+ 163us) server_negotiation.cc:407] Connection header received
0806 18:35:00.098879 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:00.098881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:00.098915 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:00.098981 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:00.099503 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42982: BlockingRecv error: recv got EOF from 127.0.0.1:42982 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260806 18:35:00.099758 18934 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/krb5cc' of type FILE
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:00.100925 18934 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:00.101821 18934 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[24795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:35:00.111187 25339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:00.102833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42990 (local address 127.18.125.129:43217)
0806 18:35:00.102977 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:00.102982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:00.103077 (+ 95us) server_negotiation.cc:407] Connection header received
0806 18:35:00.103243 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:00.103246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:00.103284 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:00.103440 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:00.103975 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.104581 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.105397 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.105630 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.105743 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:00.108045 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:00.108051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:00.108059 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:00.108082 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:00.109416 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:00.109903 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:00.109907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:00.109909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:00.109995 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:00.110280 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:00.110285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:00.110286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:00.110478 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:00.110644 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:00.110868 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 18:35:00.110988 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 18:35:00.114151 25339 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:00.113217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43004 (local address 127.18.125.129:43217)
0806 18:35:00.113356 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:00.113359 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:00.113387 (+ 28us) server_negotiation.cc:407] Connection header received
0806 18:35:00.113487 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:00.113489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:00.113525 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:00.113594 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:00.114056 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43004: BlockingRecv error: recv got EOF from 127.0.0.1:43004 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 18:35:00.114269 18934 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041269054381-18934-0/krb5kdc/krb5cc' of type FILE
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:00.115348 18934 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[24795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:00.116297 18934 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[24795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:35:00.124912 25339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:00.117208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43020 (local address 127.18.125.129:43217)
0806 18:35:00.117356 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:00.117359 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:00.117374 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:35:00.117490 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:00.117492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:00.117525 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:00.117588 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:00.118061 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.118556 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.119299 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:00.119510 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:00.119583 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:00.121925 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:00.121928 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:00.121930 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:00.121954 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:00.123236 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:00.123654 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:00.123657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:00.123658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:00.123728 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:00.123975 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:00.123977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:00.123978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:00.124232 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:00.124364 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:00.124597 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 18:35:00.124712 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:35:00.126088 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 24908
I20260806 18:35:00.133570 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 25053
I20260806 18:35:00.143154 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 25194
I20260806 18:35:00.151137 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 24809
2026-08-06T18:35:00Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2310 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[25353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[25353](info): set up 2 sockets
Aug 06 18:35:00 dist-test-slave-8w9v krb5kdc[25353](info): commencing operation
krb5kdc: starting...
W20260806 18:35:01.230566 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:35:01 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:35:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:01Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.190.keytab.
Entry for principal oryx/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.190.keytab.
I20260806 18:35:01.363420 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:42437
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.190.keytab
--principal=oryx/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:42437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:01.488220 25368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:01.488533 25368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:01.488577 25368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:01.492722 25368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:01.492787 25368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:01.492807 25368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:01.492825 25368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:01.492841 25368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:01.498271 25368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:42437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.190.keytab
--principal=oryx/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:42437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:01.499423 25368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:01.500502 25368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:01.509369 25373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:01.509349 25376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:01.509438 25368 server_base.cc:1034] running on GCE node
W20260806 18:35:01.509364 25374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:01.510335 25368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:01.511641 25368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:01.512859 25368 hybrid_clock.cc:648] HybridClock initialized: now 1786041301512844 us; error 36 us; skew 500 ppm
Aug 06 18:35:01 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041301, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:01.515815 25368 init.cc:377] Logged in from keytab as oryx/127.18.125.190@KRBTEST.COM (short username oryx)
I20260806 18:35:01.516865 25368 webserver.cc:460] Webserver started at http://127.18.125.190:33619/ using document root <none> and password file <none>
I20260806 18:35:01.517503 25368 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:01.517601 25368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:01.517850 25368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:01.520188 25368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "630b9ddf4d484387a5769bb7c7f07d5c"
format_stamp: "Formatted at 2026-08-06 18:35:01 on dist-test-slave-8w9v"
server_key: "dfe6a0ae10d2d0325e3afb7374ac8c60"
server_key_iv: "f1ea46b31f087f4448f6648642f992f1"
server_key_version: "encryptionkey@0"
I20260806 18:35:01.520834 25368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "630b9ddf4d484387a5769bb7c7f07d5c"
format_stamp: "Formatted at 2026-08-06 18:35:01 on dist-test-slave-8w9v"
server_key: "dfe6a0ae10d2d0325e3afb7374ac8c60"
server_key_iv: "f1ea46b31f087f4448f6648642f992f1"
server_key_version: "encryptionkey@0"
I20260806 18:35:01.524981 25368 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:35:01.527810 25384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:01.529224 25368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:01.529405 25368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "630b9ddf4d484387a5769bb7c7f07d5c"
format_stamp: "Formatted at 2026-08-06 18:35:01 on dist-test-slave-8w9v"
server_key: "dfe6a0ae10d2d0325e3afb7374ac8c60"
server_key_iv: "f1ea46b31f087f4448f6648642f992f1"
server_key_version: "encryptionkey@0"
I20260806 18:35:01.529543 25368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:01.544736 25368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:01.547957 25368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:01.548215 25368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:01.555995 25368 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:42437
I20260806 18:35:01.556340 25441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:42437 every 8 connection(s)
I20260806 18:35:01.557130 25368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:35:01.560110 25442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:01.561021 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 25368
I20260806 18:35:01.561131 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:35:01.561638 18934 external_mini_cluster.cc:1428] Setting key f5cc8a843af8fa187410d1595e86a64a
I20260806 18:35:01.567293 25442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c: Bootstrap starting.
I20260806 18:35:01.570428 25442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c: Neither blocks nor log segments found. Creating new log.
Aug 06 18:35:01 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
I20260806 18:35:01.571520 25442 log.cc:824] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:01.574496 25442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c: No bootstrap required, opened a new log
I20260806 18:35:01.578246 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:01.563095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.18.125.190:42437)
0806 18:35:01.563550 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:01.563565 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:01.563602 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:35:01.564343 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:01.564362 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:01.564699 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:01.564992 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:01.565940 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.566949 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.568197 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.568733 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.568880 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:01.572031 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:01.572047 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:01.572064 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:01.572102 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:01.574326 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:01.574882 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:01.574890 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:01.574897 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:01.575069 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:01.575445 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:01.575448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:01.575450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:01.576098 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:01.576353 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:01.576667 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 18:35:01.577080 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":155,"threads_started":1}
I20260806 18:35:01.579854 25442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } }
I20260806 18:35:01.580111 25442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:01.580263 25442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 630b9ddf4d484387a5769bb7c7f07d5c, State: Initialized, Role: FOLLOWER
I20260806 18:35:01.580886 25442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } }
I20260806 18:35:01.581079 25442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:35:01.581163 25442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:35:01.581290 25442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:01.582500 25442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } }
I20260806 18:35:01.582916 25442 leader_election.cc:302] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 630b9ddf4d484387a5769bb7c7f07d5c; no voters:
I20260806 18:35:01.583351 25442 leader_election.cc:288] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:35:01.583657 25447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:01.583884 25447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 1 LEADER]: Becoming Leader. State: Replica: 630b9ddf4d484387a5769bb7c7f07d5c, State: Running, Role: LEADER
I20260806 18:35:01.584300 25447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } }
I20260806 18:35:01.584911 25442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:01.586119 25448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } }
I20260806 18:35:01.586277 25448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:01.586730 25449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 630b9ddf4d484387a5769bb7c7f07d5c. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } }
I20260806 18:35:01.586836 25449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:01.587706 25456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:01.590580 25456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:01.598145 25456 catalog_manager.cc:1347] Generated new cluster ID: e60b7aa24ba2482a9ef3925ff72b8032
I20260806 18:35:01.598237 25456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:01.612282 25456 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:35:01.613566 25456 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:01.619020 25456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c: Generated new TSK 0
I20260806 18:35:01.619760 25456 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab.
Entry for principal oryx/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab.
I20260806 18:35:01.656677 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42437
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab
--principal=oryx/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:01.813714 25468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:01.814014 25468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:01.814061 25468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:01.818163 25468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:01.818228 25468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:01.818290 25468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:35:01.823700 25468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab
--principal=oryx/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:01.824867 25468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:01.825979 25468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:01.839555 25473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:01.839617 25474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:01.839556 25476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:01.840147 25468 server_base.cc:1034] running on GCE node
I20260806 18:35:01.840598 25468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:01.841460 25468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:01.842655 25468 hybrid_clock.cc:648] HybridClock initialized: now 1786041301842631 us; error 38 us; skew 500 ppm
Aug 06 18:35:01 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041301, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:01.846740 25468 init.cc:377] Logged in from keytab as oryx/127.18.125.129@KRBTEST.COM (short username oryx)
I20260806 18:35:01.848153 25468 webserver.cc:460] Webserver started at http://127.18.125.129:40867/ using document root <none> and password file <none>
I20260806 18:35:01.848819 25468 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:01.848930 25468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:01.849210 25468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:01.852286 25468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "f443fc0a32ec415297569cab7e77e2a2"
format_stamp: "Formatted at 2026-08-06 18:35:01 on dist-test-slave-8w9v"
server_key: "241bf29e937ca9b2d2a0edc3c573eb05"
server_key_iv: "a1f1d95bad61732fae2ce5a286e276f8"
server_key_version: "encryptionkey@0"
I20260806 18:35:01.853089 25468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "f443fc0a32ec415297569cab7e77e2a2"
format_stamp: "Formatted at 2026-08-06 18:35:01 on dist-test-slave-8w9v"
server_key: "241bf29e937ca9b2d2a0edc3c573eb05"
server_key_iv: "a1f1d95bad61732fae2ce5a286e276f8"
server_key_version: "encryptionkey@0"
I20260806 18:35:01.858695 25468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 18:35:01.862498 25483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:01.864346 25468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:01.864471 25468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "f443fc0a32ec415297569cab7e77e2a2"
format_stamp: "Formatted at 2026-08-06 18:35:01 on dist-test-slave-8w9v"
server_key: "241bf29e937ca9b2d2a0edc3c573eb05"
server_key_iv: "a1f1d95bad61732fae2ce5a286e276f8"
server_key_version: "encryptionkey@0"
I20260806 18:35:01.864599 25468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:01.878096 25468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:01.882789 25468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:01.883030 25468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:01.883874 25468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:01.884904 25468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:01.884958 25468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:01.884999 25468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:01.885015 25468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:01.894259 25468 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:41241
I20260806 18:35:01.894284 25605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:41241 every 8 connection(s)
I20260806 18:35:01.895257 25468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:01.895634 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 25468
I20260806 18:35:01.895771 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:35:01.896191 18934 external_mini_cluster.cc:1428] Setting key 0e31d8b4b9568398f88ac7e9ef59c12f
Aug 06 18:35:01 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041301, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.130@KRBTEST.COM" created.
I20260806 18:35:01.912812 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:01.897462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:40135 (local address 127.18.125.190:42437)
0806 18:35:01.897585 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:01.897589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:01.898773 (+ 1184us) server_negotiation.cc:407] Connection header received
0806 18:35:01.900760 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:01.900765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:01.900837 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:01.900938 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:01.902140 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.902728 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.904675 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.904941 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.905039 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:01.908623 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:01.908626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:01.908628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:01.908657 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:01.910244 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:01.910804 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:01.910806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:01.910808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:01.910936 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:01.911279 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:01.911282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:01.911283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:01.911480 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:01.911627 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:01.912393 (+ 766us) server_negotiation.cc:299] Negotiation successful
0806 18:35:01.912554 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:35:01.913633 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:01.898032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.129:40135)
0806 18:35:01.898606 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:01.898670 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:01.900527 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:01.901101 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:01.901108 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:01.901496 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:01.901988 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:01.902002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.902898 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.902905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:01.904498 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:01.904515 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:01.905142 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:01.905149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:01.905416 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:01.906174 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:01.906202 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:01.908412 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:01.910381 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:01.910387 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:01.910401 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:01.910695 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:01.911030 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:01.911032 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:01.911034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:01.911181 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:01.911626 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:01.911661 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:01.912037 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 18:35:01.912304 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 18:35:01.912604 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":126,"threads_started":1}
I20260806 18:35:01.915058 25606 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42437
I20260806 18:35:01.915356 25606 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:01.915970 25606 heartbeater.cc:511] Master 127.18.125.190:42437 requested a full tablet report, sending...
I20260806 18:35:01.917678 25403 ts_manager.cc:194] Registered new tserver with Master: f443fc0a32ec415297569cab7e77e2a2 (127.18.125.129:41241)
I20260806 18:35:01.918862 25403 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:40135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab.
Entry for principal oryx/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab.
I20260806 18:35:01.931470 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42437
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab
--principal=oryx/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:02.053346 25611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:02.053719 25611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:02.053808 25611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:02.058136 25611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:02.058260 25611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:02.058372 25611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:35:02.064072 25611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab
--principal=oryx/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:02.065336 25611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:02.066488 25611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:02.077507 25619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:02.077554 25616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:02.077515 25617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:02.077530 25611 server_base.cc:1034] running on GCE node
I20260806 18:35:02.078239 25611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:02.078984 25611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:02.080192 25611 hybrid_clock.cc:648] HybridClock initialized: now 1786041302080180 us; error 28 us; skew 500 ppm
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:02.083139 25611 init.cc:377] Logged in from keytab as oryx/127.18.125.130@KRBTEST.COM (short username oryx)
I20260806 18:35:02.084056 25611 webserver.cc:460] Webserver started at http://127.18.125.130:33079/ using document root <none> and password file <none>
I20260806 18:35:02.084728 25611 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:02.084796 25611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:02.085045 25611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:02.087303 25611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "dd88c02290fb40238ca877dda174f5a9"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "fd987e455ea803fe7281c4202811ad5b"
server_key_iv: "65d61a0be6b6807d9f09bdec5c05e975"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.087883 25611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "dd88c02290fb40238ca877dda174f5a9"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "fd987e455ea803fe7281c4202811ad5b"
server_key_iv: "65d61a0be6b6807d9f09bdec5c05e975"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.091859 25611 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:35:02.094486 25626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:02.095806 25611 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:35:02.095944 25611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "dd88c02290fb40238ca877dda174f5a9"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "fd987e455ea803fe7281c4202811ad5b"
server_key_iv: "65d61a0be6b6807d9f09bdec5c05e975"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.096083 25611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:02.130029 25611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:02.133919 25611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:02.134176 25611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:02.134931 25611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:02.135881 25611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:02.135973 25611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:02.136054 25611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:02.136111 25611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:02.145542 25611 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:33231
I20260806 18:35:02.145566 25753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:33231 every 8 connection(s)
I20260806 18:35:02.146481 25611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:02.149813 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 25611
I20260806 18:35:02.149953 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:35:02.150245 18934 external_mini_cluster.cc:1428] Setting key d7b2546f748229d458abee0a023b8771
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:35:02.164077 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.148688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:59061 (local address 127.18.125.190:42437)
0806 18:35:02.149000 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.149007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.149791 (+ 784us) server_negotiation.cc:407] Connection header received
0806 18:35:02.151587 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.151590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.151645 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.151742 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.153502 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.154106 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.155766 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.156228 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.156366 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.159317 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:02.159320 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:02.159323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:02.159369 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:02.161146 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.161841 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.161845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.161846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.161937 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.162757 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.162760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.162762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.162978 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:02.163071 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:02.163760 (+ 689us) server_negotiation.cc:299] Negotiation successful
0806 18:35:02.163894 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.131@KRBTEST.COM" created.
I20260806 18:35:02.165241 25756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.149058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.130:59061)
0806 18:35:02.149598 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.149650 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.151241 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.151958 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.151968 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.152561 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.153319 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.153345 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.154259 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.154266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.155612 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.155623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.156408 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.156411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.156564 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.157236 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:02.157254 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:02.159082 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:02.161313 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.161321 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.161341 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.161701 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.162185 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.162190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.162192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.162609 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.163132 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:02.163139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:02.163444 (+ 305us) client_negotiation.cc:769] Sending connection context
0806 18:35:02.163640 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 18:35:02.164072 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":129,"threads_started":1}
I20260806 18:35:02.166632 25754 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42437
I20260806 18:35:02.166975 25754 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:02.167691 25754 heartbeater.cc:511] Master 127.18.125.190:42437 requested a full tablet report, sending...
I20260806 18:35:02.169066 25403 ts_manager.cc:194] Registered new tserver with Master: dd88c02290fb40238ca877dda174f5a9 (127.18.125.130:33231)
I20260806 18:35:02.169770 25403 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:59061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab.
Entry for principal oryx/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab.
I20260806 18:35:02.185250 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42437
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab
--principal=oryx/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:02.306766 25759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:02.307098 25759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:02.307183 25759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:02.311662 25759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:02.311775 25759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:02.311887 25759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:35:02.318002 25759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab
--principal=oryx/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:02.319238 25759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:02.320382 25759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:02.330886 25764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:02.330981 25759 server_base.cc:1034] running on GCE node
W20260806 18:35:02.330881 25767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:02.330881 25765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:02.331570 25759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:02.332278 25759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:02.333520 25759 hybrid_clock.cc:648] HybridClock initialized: now 1786041302333499 us; error 47 us; skew 500 ppm
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:02.336618 25759 init.cc:377] Logged in from keytab as oryx/127.18.125.131@KRBTEST.COM (short username oryx)
I20260806 18:35:02.337654 25759 webserver.cc:460] Webserver started at http://127.18.125.131:39115/ using document root <none> and password file <none>
I20260806 18:35:02.338299 25759 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:02.338400 25759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:02.338654 25759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:02.341045 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "fe73b370c34842228aa10e9f0a150438"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "3996563f16fcb46002eeaa8daccfec12"
server_key_iv: "170298862b7cfa7844b8803c4192b20b"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.341661 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "fe73b370c34842228aa10e9f0a150438"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "3996563f16fcb46002eeaa8daccfec12"
server_key_iv: "170298862b7cfa7844b8803c4192b20b"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.345750 25759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:35:02.348416 25774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:02.349669 25759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:35:02.349838 25759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "fe73b370c34842228aa10e9f0a150438"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "3996563f16fcb46002eeaa8daccfec12"
server_key_iv: "170298862b7cfa7844b8803c4192b20b"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.349968 25759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:02.367026 25759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:02.370579 25759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:02.370812 25759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:02.371483 25759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:02.372480 25759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:02.372552 25759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:02.372625 25759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:02.372663 25759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:02.383947 25759 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:36445
I20260806 18:35:02.383971 25895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:36445 every 8 connection(s)
I20260806 18:35:02.385106 25759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:02.393278 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 25759
I20260806 18:35:02.393440 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:35:02.393746 18934 external_mini_cluster.cc:1428] Setting key 13bc7c153cd69e4a28c480a786e5c638
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
I20260806 18:35:02.399986 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.387224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:42719 (local address 127.18.125.190:42437)
0806 18:35:02.387382 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.387386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.388347 (+ 961us) server_negotiation.cc:407] Connection header received
0806 18:35:02.389774 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.389779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.389838 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.389923 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.391136 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.391684 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.392960 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.393192 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.393315 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.396579 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:02.396583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:02.396586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:02.396616 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:02.397979 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.398481 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.398484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.398485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.398563 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.398889 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.398891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.398892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.399096 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:02.399197 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:02.399754 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 18:35:02.399862 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:35:02.400907 25899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.387645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.131:42719)
0806 18:35:02.388184 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.388228 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.389546 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.390107 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.390115 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.390486 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.390975 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.390989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.391862 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.391865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.392801 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.392812 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.393364 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.393368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.393617 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.394340 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:02.394368 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:02.396399 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:02.398092 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.398098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.398110 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.398381 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.398644 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.398646 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.398648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.398801 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.399201 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:02.399206 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:02.399502 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 18:35:02.399684 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 18:35:02.399962 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":151,"threads_started":1}
I20260806 18:35:02.402135 25896 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42437
I20260806 18:35:02.402365 25896 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:02.402853 25896 heartbeater.cc:511] Master 127.18.125.190:42437 requested a full tablet report, sending...
I20260806 18:35:02.403910 25403 ts_manager.cc:194] Registered new tserver with Master: fe73b370c34842228aa10e9f0a150438 (127.18.125.131:36445)
I20260806 18:35:02.404551 25403 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:42719
I20260806 18:35:02.408192 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.125.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.189.keytab.
Entry for principal oryx/127.18.125.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.189.keytab.
I20260806 18:35:02.441056 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.189:41833
--webserver_interface=127.18.125.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.189.keytab
--principal=oryx/127.18.125.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:42437,127.18.125.189:41833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:02.563941 25902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:02.564358 25902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:02.564440 25902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:02.570685 25902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:02.570776 25902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:02.570811 25902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:02.570842 25902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:02.570858 25902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:02.576866 25902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:42437,127.18.125.189:41833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.189.keytab
--principal=oryx/127.18.125.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.189:41833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.125.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
W20260806 18:35:02.577108 25902 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.125.190:42437,127.18.125.189:41833'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 18:35:02.578045 25902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:02.579070 25902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:02.588354 25907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:02.588425 25902 server_base.cc:1034] running on GCE node
W20260806 18:35:02.588354 25910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:02.588685 25908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:02.589293 25902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:02.590468 25902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:02.591661 25902 hybrid_clock.cc:648] HybridClock initialized: now 1786041302591622 us; error 48 us; skew 500 ppm
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:02.594664 25902 init.cc:377] Logged in from keytab as oryx/127.18.125.189@KRBTEST.COM (short username oryx)
I20260806 18:35:02.595567 25902 webserver.cc:460] Webserver started at http://127.18.125.189:44447/ using document root <none> and password file <none>
I20260806 18:35:02.596227 25902 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:02.596297 25902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:02.596481 25902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:02.598664 25902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data/instance:
uuid: "c68e5972701347c3a3f2c2516c7313b5"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "ba5f490bcc263a694fa4cee24d99bc91"
server_key_iv: "4d13877d686a5dd458c60dc1bcba6ed2"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.599278 25902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/wal/instance:
uuid: "c68e5972701347c3a3f2c2516c7313b5"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "ba5f490bcc263a694fa4cee24d99bc91"
server_key_iv: "4d13877d686a5dd458c60dc1bcba6ed2"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.603184 25902 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:35:02.605820 25917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:02.607132 25902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:35:02.607287 25902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/wal
uuid: "c68e5972701347c3a3f2c2516c7313b5"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "ba5f490bcc263a694fa4cee24d99bc91"
server_key_iv: "4d13877d686a5dd458c60dc1bcba6ed2"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.607421 25902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:02.626080 25902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:02.629415 25902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:02.629647 25902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:02.637271 25902 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.189:41833
I20260806 18:35:02.637570 25971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.189:41833 every 8 connection(s)
I20260806 18:35:02.638375 25902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
I20260806 18:35:02.638913 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 25902
I20260806 18:35:02.642438 25972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:02.646515 25972 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.125.189@KRBTEST.COM
I20260806 18:35:02.654412 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.640480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.18.125.189:41833)
0806 18:35:02.640906 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.640922 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.640960 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:35:02.641630 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.641646 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.641993 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.642346 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.643175 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.644291 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.645161 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.645656 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.645849 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.648718 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:02.648726 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:02.648743 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:02.648779 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:02.651063 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.651646 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.651650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.651655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.651769 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.652068 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.652070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.652071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.652632 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:02.652850 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:02.653129 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 18:35:02.653442 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":115,"threads_started":1}
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.189@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
I20260806 18:35:02.659119 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.648578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53194 (local address 127.18.125.190:42437)
0806 18:35:02.648748 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.648752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.649119 (+ 367us) server_negotiation.cc:407] Connection header received
0806 18:35:02.649293 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.649297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.649356 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.649446 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.650319 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.651151 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.652410 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.652743 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.652850 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.655564 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:02.655567 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:02.655569 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:02.655597 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:02.657296 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.657819 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.657822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.657823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.657905 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.658237 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.658241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.658242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.658502 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:02.658593 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:02.658894 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 18:35:02.659003 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:35:02.659149 25976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.648517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:53194)
0806 18:35:02.648979 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.649023 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.649184 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.649505 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.649510 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.649976 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.650173 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.650181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.651267 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.651273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.652292 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.652303 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.652862 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.652866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.653034 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.653642 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:02.653655 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:02.655401 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:02.657422 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.657429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.657436 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.657739 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.657990 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.657992 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.657994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.658073 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.658602 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:02.658607 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:02.658713 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 18:35:02.658826 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 18:35:02.658990 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":158,"threads_started":1}
I20260806 18:35:02.660485 25972 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.189@KRBTEST.COM for oryx/127.18.125.189@KRBTEST.COM
I20260806 18:35:02.668691 25976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.661006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:41833 (local address 127.0.0.1:47416)
0806 18:35:02.661128 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.661143 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.661220 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.661421 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.661423 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.661550 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.661653 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.661658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.662833 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.662836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.663369 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.663378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.663828 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.663831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.663905 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.664485 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:02.664503 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:02.665737 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:02.667279 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.667282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.667283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.667487 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.667737 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.667739 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.667741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.667816 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.668225 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:02.668228 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:02.668317 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 18:35:02.668424 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 18:35:02.668535 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 18:35:02.668711 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.661084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47416 (local address 127.18.125.189:41833)
0806 18:35:02.661212 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.661215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.661228 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:35:02.661290 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.661292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.661326 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.661393 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.661748 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.662697 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.663481 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.663725 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.663808 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.665865 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:02.665868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:02.665870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:02.665898 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:02.667157 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.667579 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.667582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.667583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.667654 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.667900 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.667902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.667903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.668073 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:02.668214 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:02.668484 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 18:35:02.668586 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:35:02.671955 25972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: Bootstrap starting.
I20260806 18:35:02.674475 25972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:02.675263 25972 log.cc:824] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:02.678012 25972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: No bootstrap required, opened a new log
I20260806 18:35:02.681293 25972 raft_consensus.cc:348] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } }
I20260806 18:35:02.681524 25972 raft_consensus.cc:374] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:02.681617 25972 raft_consensus.cc:729] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c68e5972701347c3a3f2c2516c7313b5, State: Initialized, Role: FOLLOWER
I20260806 18:35:02.682096 25972 consensus_queue.cc:260] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } }
I20260806 18:35:02.682636 25978 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } } }
I20260806 18:35:02.682776 25978 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:35:02.683404 25972 sys_catalog.cc:564] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 18:35:02.688715 25989 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:35:02.688824 25989 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:35:02.690874 25902 master_runner.cc:428] Detected that this master c68e5972701347c3a3f2c2516c7313b5 is joining an existing cluster
I20260806 18:35:02.690932 25902 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.125.189:41833
I20260806 18:35:02.703258 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.693806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.18.125.189:41833)
0806 18:35:02.693912 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.693916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.693981 (+ 65us) server_negotiation.cc:407] Connection header received
0806 18:35:02.694083 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.694086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.694133 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.694219 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.695769 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.696593 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.697625 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.697985 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.698099 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.699403 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:02.699409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:02.699417 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:02.699436 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:02.701354 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.701949 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.701953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.701955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.702031 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.702305 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.702308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.702310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.702493 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:02.702600 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:02.702861 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 18:35:02.703039 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":42}
I20260806 18:35:02.703279 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.693185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.18.125.190:42437)
0806 18:35:02.693332 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.693336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.693617 (+ 281us) server_negotiation.cc:407] Connection header received
0806 18:35:02.693817 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.693820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.693873 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.693937 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.694743 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.695609 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.696552 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.696884 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.696994 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.699576 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:02.699583 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:02.699586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:02.699622 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:02.701385 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.701888 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.701891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.701893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.701990 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.702320 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.702324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.702326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.702556 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:02.702628 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:02.703003 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 18:35:02.703123 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:35:02.703342 25999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.693562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:41833 (local address 127.0.0.1:47426)
0806 18:35:02.693880 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.693893 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.693991 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.694248 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.694252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.694480 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.694610 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.694625 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.696722 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.696725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.697493 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.697504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.698116 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.698119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.698199 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.698739 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:02.698750 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:02.699261 (+ 511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:02.701481 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.701485 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.701487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.701819 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.702129 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.702132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.702134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.702209 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.702589 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:02.702592 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:02.702699 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 18:35:02.702811 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:35:02.703153 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":4,"spinlock_wait_cycles":3712,"thread_start_us":88,"threads_started":1}
I20260806 18:35:02.704272 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.693114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:53204)
0806 18:35:02.693535 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.693551 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.693663 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.694094 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.694097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.694434 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.694605 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.694616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.695729 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.695731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.696418 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.696427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.697031 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.697035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.697139 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.698745 (+ 1606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:02.698761 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:02.699425 (+ 664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:02.701502 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.701505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.701507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.701774 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.702098 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.702101 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.702103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.702200 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.702702 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:02.702705 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:02.702795 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 18:35:02.702933 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 18:35:02.703118 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":22,"thread_start_us":109,"threads_started":1}
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041301, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.190@KRBTEST.COM for oryx/127.18.125.189@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.125.188@KRBTEST.COM; defaulting to no policy
I20260806 18:35:02.719873 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.709496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47436 (local address 127.18.125.189:41833)
0806 18:35:02.709618 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.709622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.709635 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:35:02.709699 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.709702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.709750 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.709832 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.710758 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.711326 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.712492 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.712833 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.712950 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.715998 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:02.716002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:02.716005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:02.716034 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:02.717884 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.718556 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.718562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.718566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.718658 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.718959 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.718961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.718962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.719145 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:02.719220 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:02.719647 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 18:35:02.719746 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:35:02.720007 26000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.708755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:41833 (local address 127.0.0.1:47436)
0806 18:35:02.709137 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.709192 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.709340 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.709896 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.709906 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.710417 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.710597 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.710608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.711494 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.711497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.712364 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.712378 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.712963 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.712971 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.713117 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.713811 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:02.713832 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:02.715817 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:02.718015 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.718027 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.718035 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.718460 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.718766 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.718770 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.718772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.718869 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.719251 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:02.719258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:02.719402 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 18:35:02.719600 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 18:35:02.719814 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.188@KRBTEST.COM" created.
I20260806 18:35:02.721692 25403 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.125.189:41833: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5"
member_type: NON_VOTER
last_known_addr {
host: "127.18.125.189"
port: 41833
}
attrs {
promote: true
}
}
dest_uuid: "630b9ddf4d484387a5769bb7c7f07d5c"
cas_config_opid_index: -1
I20260806 18:35:02.722380 25403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } }
I20260806 18:35:02.724586 26004 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c68e5972701347c3a3f2c2516c7313b5 (127.18.125.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } } }
I20260806 18:35:02.725303 25945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:02.726387 26004 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.125.189:41833
I20260806 18:35:02.726634 26004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } } }
I20260806 18:35:02.726718 26004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:02.726760 25945 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Refusing update from remote peer 630b9ddf4d484387a5769bb7c7f07d5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 18:35:02.726884 25902 master.cc:537] Master@127.18.125.189:41833 shutting down...
I20260806 18:35:02.727572 26004 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:35:02.730202 25386 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 18:35:02.731205 25386 consensus_peers.cc:487] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c -> Peer c68e5972701347c3a3f2c2516c7313b5 (127.18.125.189:41833): Couldn't send request to peer c68e5972701347c3a3f2c2516c7313b5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:35:02.732820 25902 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:02.733135 25902 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:02.733251 25902 tablet_replica.cc:330] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.188.keytab.
Entry for principal oryx/127.18.125.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.188.keytab.
I20260806 18:35:02.744304 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.188:38057
--webserver_interface=127.18.125.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.188.keytab
--principal=oryx/127.18.125.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:42437,127.18.125.189:41833,127.18.125.188:38057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:35:02.748517 25902 master.cc:552] Master@127.18.125.189:41833 shutdown complete.
I20260806 18:35:02.748587 25902 master_runner.cc:305] Clearing existing system tablet
I20260806 18:35:02.748862 25902 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:02.748932 25902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:02.750396 25902 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 18:35:02.751415 26010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:02.751745 25902 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:02.751919 25902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:35:02.751951 25902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/wal
uuid: "c68e5972701347c3a3f2c2516c7313b5"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "ba5f490bcc263a694fa4cee24d99bc91"
server_key_iv: "4d13877d686a5dd458c60dc1bcba6ed2"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.754329 25902 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:02.757193 25902 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 18:35:02.757295 25902 log.cc:1196] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 18:35:02.757570 25902 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: Deleting consensus metadata
I20260806 18:35:02.757745 25902 master_runner.cc:315] Copying system tablet from 127.18.125.190:42437
I20260806 18:35:02.759191 25902 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: tablet copy: Beginning tablet copy session from remote peer at address 127.18.125.190:42437
I20260806 18:35:02.769395 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.759873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:53212)
0806 18:35:02.760277 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.760292 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.760434 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.760742 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.760745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.760960 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.761115 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.761123 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.762304 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.762308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.763237 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.763250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.764096 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.764104 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.764250 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.764897 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:02.764913 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:02.765376 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:02.767475 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.767479 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.767481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.767744 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.768070 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:02.768073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:02.768075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:02.768204 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:02.768724 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:02.768727 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:02.768838 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 18:35:02.768944 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 18:35:02.769102 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":88,"threads_started":1}
I20260806 18:35:02.769418 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.759965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53212 (local address 127.18.125.190:42437)
0806 18:35:02.760104 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.760109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.760413 (+ 304us) server_negotiation.cc:407] Connection header received
0806 18:35:02.760556 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.760559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.760619 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.760701 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.761247 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.762182 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.763423 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.763830 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.764101 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:02.765546 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:02.765552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:02.765556 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:02.765587 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:02.767331 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.767854 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.767858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.767860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.767958 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:02.768317 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:02.768320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:02.768322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:02.768621 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:02.768723 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:02.769038 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 18:35:02.769173 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:35:02.770457 25425 tablet_copy_service.cc:140] P 630b9ddf4d484387a5769bb7c7f07d5c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c68e5972701347c3a3f2c2516c7313b5 ({username='oryx', principal='oryx/127.18.125.189@KRBTEST.COM'} at 127.0.0.1:53212)
I20260806 18:35:02.770653 25425 tablet_copy_service.cc:161] P 630b9ddf4d484387a5769bb7c7f07d5c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c68e5972701347c3a3f2c2516c7313b5 at {username='oryx', principal='oryx/127.18.125.189@KRBTEST.COM'} at 127.0.0.1:53212: session id = c68e5972701347c3a3f2c2516c7313b5-00000000000000000000000000000000
I20260806 18:35:02.772735 25425 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:35:02.774683 25902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 18:35:02.778374 25902 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: tablet copy: Starting download of 0 data blocks...
I20260806 18:35:02.778910 25902 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: tablet copy: Starting download of 1 WAL segments...
I20260806 18:35:02.781599 25902 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:35:02.784940 25425 tablet_copy_service.cc:342] P 630b9ddf4d484387a5769bb7c7f07d5c: Request end of tablet copy session c68e5972701347c3a3f2c2516c7313b5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.125.189@KRBTEST.COM'} at 127.0.0.1:53212
I20260806 18:35:02.785187 25425 tablet_copy_service.cc:433] P 630b9ddf4d484387a5769bb7c7f07d5c: ending tablet copy session c68e5972701347c3a3f2c2516c7313b5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c68e5972701347c3a3f2c2516c7313b5
I20260806 18:35:02.794390 25902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:02.796571 26018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:02.796602 26021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:02.797019 26019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:02.797314 25902 server_base.cc:1034] running on GCE node
I20260806 18:35:02.797478 25902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:02.797782 25902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:02.799167 25902 hybrid_clock.cc:648] HybridClock initialized: now 1786041302799142 us; error 44 us; skew 500 ppm
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:02.800863 25902 init.cc:377] Logged in from keytab as oryx/127.18.125.189@KRBTEST.COM (short username oryx)
I20260806 18:35:02.801344 25902 webserver.cc:460] Webserver started at http://127.18.125.189:45401/ using document root <none> and password file <none>
I20260806 18:35:02.801548 25902 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:02.801591 25902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:02.802969 25902 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 18:35:02.804049 26028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:02.804351 25902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 18:35:02.804425 25902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/wal
uuid: "c68e5972701347c3a3f2c2516c7313b5"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "ba5f490bcc263a694fa4cee24d99bc91"
server_key_iv: "4d13877d686a5dd458c60dc1bcba6ed2"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.804500 25902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:02.811452 25902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:02.813838 25902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:02.820183 25902 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.189:41833
I20260806 18:35:02.820815 26083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.189:41833 every 8 connection(s)
I20260806 18:35:02.820968 25902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
I20260806 18:35:02.822932 26084 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:35:02.824093 26084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: Bootstrap starting.
I20260806 18:35:02.836915 26084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:35:02.837340 26084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: Bootstrap complete.
I20260806 18:35:02.838014 26084 raft_consensus.cc:348] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } }
I20260806 18:35:02.838191 26084 raft_consensus.cc:729] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c68e5972701347c3a3f2c2516c7313b5, State: Initialized, Role: LEARNER
I20260806 18:35:02.838308 26084 consensus_queue.cc:260] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } }
I20260806 18:35:02.838864 26084 sys_catalog.cc:564] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:02.839020 26087 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } } }
I20260806 18:35:02.839265 26087 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:35:02.842344 26099 catalog_manager.cc:1259] Loaded cluster ID: e60b7aa24ba2482a9ef3925ff72b8032
I20260806 18:35:02.842708 26099 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: loading cluster ID for follower catalog manager: success
I20260806 18:35:02.845115 26099 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: acquiring CA information for follower catalog manager: success
I20260806 18:35:02.846690 26099 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 18:35:02.888170 26008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:02.888516 26008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:02.888633 26008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:02.892850 26008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:02.892956 26008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:02.892998 26008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:02.893065 26008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:02.893127 26008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:02.899019 26008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:42437,127.18.125.189:41833,127.18.125.188:38057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.188.keytab
--principal=oryx/127.18.125.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.188:38057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.125.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:02.900300 26008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:02.901445 26008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:02.910420 26104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:02.910529 26107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:02.910416 26105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:02.910482 26008 server_base.cc:1034] running on GCE node
I20260806 18:35:02.911317 26008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:02.912549 26008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:02.913728 26008 hybrid_clock.cc:648] HybridClock initialized: now 1786041302913691 us; error 49 us; skew 500 ppm
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:02.916683 26008 init.cc:377] Logged in from keytab as oryx/127.18.125.188@KRBTEST.COM (short username oryx)
I20260806 18:35:02.917604 26008 webserver.cc:460] Webserver started at http://127.18.125.188:44471/ using document root <none> and password file <none>
I20260806 18:35:02.918162 26008 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:02.918221 26008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:02.918393 26008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:02.920636 26008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data/instance:
uuid: "2b4c9cd7083b46b3b9d9df66473a4846"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "0fb2ef45b2e48796bc3795315230abb4"
server_key_iv: "3cfab1e64c3987ae7ed30ef1d239b408"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.921234 26008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/wal/instance:
uuid: "2b4c9cd7083b46b3b9d9df66473a4846"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "0fb2ef45b2e48796bc3795315230abb4"
server_key_iv: "3cfab1e64c3987ae7ed30ef1d239b408"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.921522 25606 heartbeater.cc:503] Master 127.18.125.190:42437 was elected leader, sending a full tablet report...
I20260806 18:35:02.925395 26008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:35:02.928074 26114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:02.929369 26008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:02.929540 26008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/wal
uuid: "2b4c9cd7083b46b3b9d9df66473a4846"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "0fb2ef45b2e48796bc3795315230abb4"
server_key_iv: "3cfab1e64c3987ae7ed30ef1d239b408"
server_key_version: "encryptionkey@0"
I20260806 18:35:02.929664 26008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:02.972635 26008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:02.976246 26008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:02.976478 26008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:02.984421 26008 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.188:38057
I20260806 18:35:02.984964 26167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.188:38057 every 8 connection(s)
I20260806 18:35:02.985577 26008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
I20260806 18:35:02.988552 26168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:02.992439 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 26008
I20260806 18:35:02.992708 26168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.188@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
Aug 06 18:35:02 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.125.188@KRBTEST.COM
I20260806 18:35:03.008157 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.995591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.18.125.190:42437)
0806 18:35:02.995726 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.995731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.995978 (+ 247us) server_negotiation.cc:407] Connection header received
0806 18:35:02.996272 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.996276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.996356 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.996447 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.997357 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.998127 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.999703 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.000038 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.000205 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.003501 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.003506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.003509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.003543 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.005550 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.006150 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.006154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.006155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.006244 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.006656 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.006660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.006663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.006983 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.007094 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.007758 (+ 664us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.007928 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:35:03.009279 26170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.993970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54302 (local address 127.18.125.188:38057)
0806 18:35:02.994401 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:02.994417 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:02.994454 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:35:02.995240 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:02.995265 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:02.995688 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:02.996029 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:02.996952 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.998335 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.999562 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.000236 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.000387 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.004174 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.004183 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.004201 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.004230 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.006349 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.006835 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.006840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.006841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.006919 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.007230 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.007233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.007234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.007533 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.007764 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.008029 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.008199 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":289,"spinlock_wait_cycles":5376,"thread_start_us":132,"threads_started":1}
I20260806 18:35:03.009279 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:02.995432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:53228)
0806 18:35:02.995817 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:02.995857 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:02.996115 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:02.996542 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:02.996549 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:02.997017 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:02.997187 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.997198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:02.998287 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:02.998289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:02.999541 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:02.999554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.000324 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.000331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.000447 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.001333 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.001356 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.003295 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.005682 (+ 2387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.005694 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.005709 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.006036 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.006349 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.006361 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.006363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.006552 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.007099 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.007105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.007496 (+ 391us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.007695 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.008096 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"spinlock_wait_cycles":3840,"thread_start_us":97,"threads_started":1}
I20260806 18:35:03.011055 26168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.188@KRBTEST.COM for oryx/127.18.125.189@KRBTEST.COM
I20260806 18:35:03.020599 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.011849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47440 (local address 127.18.125.189:41833)
0806 18:35:03.012290 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.012294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.012308 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:35:03.012352 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.012354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.012397 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.012520 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.012988 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.013736 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.014612 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.014863 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.014993 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.017252 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.017255 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.017257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.017281 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.018800 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.019269 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.019278 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.019279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.019358 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.019641 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.019646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.019647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.019857 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.019969 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.020302 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.020428 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":132,"threads_started":1}
I20260806 18:35:03.020598 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.011683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:41833 (local address 127.0.0.1:47440)
0806 18:35:03.011952 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.011970 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.012066 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.012537 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.012540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.012740 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.012852 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.012858 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.013867 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.013874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.014456 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.014465 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.014992 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.014996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.015121 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.015612 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.015625 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.017091 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.018943 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.018946 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.018948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.019158 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.019455 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.019459 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.019461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.019545 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.019961 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.019964 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.020070 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.020242 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.020416 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260806 18:35:03.021489 26168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041302, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.188@KRBTEST.COM for oryx/127.18.125.188@KRBTEST.COM
I20260806 18:35:03.029599 26170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.022057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54318 (local address 127.18.125.188:38057)
0806 18:35:03.022175 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.022181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.022234 (+ 53us) server_negotiation.cc:407] Connection header received
0806 18:35:03.022396 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.022399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.022442 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.022519 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.022871 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.023388 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.024161 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.024402 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.024513 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.026650 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.026654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.026656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.026676 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.027908 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.028403 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.028408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.028410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.028483 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.028757 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.028760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.028762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.028939 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.029042 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.029290 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.029448 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":45}
I20260806 18:35:03.029599 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.022021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38057 (local address 127.0.0.1:54318)
0806 18:35:03.022175 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.022191 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.022276 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.022529 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.022532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.022699 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.022764 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.022768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.023518 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.023521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.024025 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.024031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.024511 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.024514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.024652 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.025124 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.025137 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.026517 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.028023 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.028028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.028031 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.028308 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.028573 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.028577 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.028579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.028660 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.029042 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.029045 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.029134 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.029235 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.029390 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":37}
I20260806 18:35:03.033305 26168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: Bootstrap starting.
I20260806 18:35:03.036223 26168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:03.036989 26168 log.cc:824] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:03.039191 26168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: No bootstrap required, opened a new log
I20260806 18:35:03.042547 26168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } }
I20260806 18:35:03.042759 26168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:03.042793 26168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b4c9cd7083b46b3b9d9df66473a4846, State: Initialized, Role: FOLLOWER
I20260806 18:35:03.043165 26168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } }
I20260806 18:35:03.043655 26174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } } }
I20260806 18:35:03.043812 26174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:35:03.044458 26168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 18:35:03.049678 26185 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:35:03.049788 26185 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:35:03.053609 26008 master_runner.cc:428] Detected that this master 2b4c9cd7083b46b3b9d9df66473a4846 is joining an existing cluster
I20260806 18:35:03.053733 26008 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.125.188:38057
I20260806 18:35:03.063598 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 25468
I20260806 18:35:03.068936 26191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.056508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38057 (local address 127.0.0.1:54334)
0806 18:35:03.057439 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.057453 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.057567 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.057883 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.057886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.058244 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.058637 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.058648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.059706 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.059709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.060864 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.060876 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.061772 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.061779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.061904 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.062687 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.062705 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.063205 (+ 500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.066863 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.066868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.066871 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.067240 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.067609 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.067613 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.067616 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.067746 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.068329 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.068332 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.068417 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.068531 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.068693 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":488,"spinlock_wait_cycles":2304,"thread_start_us":153,"threads_started":1}
I20260806 18:35:03.068986 26170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.056631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54334 (local address 127.18.125.188:38057)
0806 18:35:03.056760 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.056765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.057515 (+ 750us) server_negotiation.cc:407] Connection header received
0806 18:35:03.057645 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.057650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.057697 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.057768 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.058750 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.059570 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.060998 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.061423 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.061547 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.063346 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.063350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.063353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.063377 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.066680 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.067384 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.067388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.067390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.067492 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.067875 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.067879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.067881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.068177 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.068291 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.068620 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.068807 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":51}
I20260806 18:35:03.081797 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:41241
--local_ip_for_outbound_sockets=127.18.125.129
--tserver_master_addrs=127.18.125.190:42437
--webserver_port=40867
--webserver_interface=127.18.125.129
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab
--principal=oryx/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.125.190:42437,127.18.125.189:41833
--tserver_master_addrs=127.18.125.190:42437,127.18.125.189:41833,127.18.125.188:38057 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:35:03.082502 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.056535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53230 (local address 127.18.125.190:42437)
0806 18:35:03.056678 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.056683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.057202 (+ 519us) server_negotiation.cc:407] Connection header received
0806 18:35:03.057699 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.057703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.057762 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.057813 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.058741 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.059570 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.061011 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.061381 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.061455 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.077898 (+ 16443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.077903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.077906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.077942 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.080165 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.080802 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.080806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.080808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.080920 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.081399 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.081403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.081404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.081638 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.081716 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.082170 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.082304 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:35:03.083142 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.056427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:53230)
0806 18:35:03.057100 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.057115 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.057250 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.057968 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.057973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.058244 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.058582 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.058594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.059706 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.059709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.060863 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.060878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.061877 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.061880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.061959 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.076931 (+ 14972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.076945 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.077660 (+ 715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.080282 (+ 2622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.080285 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.080287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.080664 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.081181 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.081186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.081191 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.081273 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.081798 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.081801 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.081964 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.082719 (+ 755us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.082917 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":180,"spinlock_wait_cycles":12160,"thread_start_us":153,"threads_started":1}
I20260806 18:35:03.083518 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.057377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47448 (local address 127.18.125.189:41833)
0806 18:35:03.057527 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.057533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.057802 (+ 269us) server_negotiation.cc:407] Connection header received
0806 18:35:03.057994 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.057997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.058049 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.058105 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.059314 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.060154 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.061780 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.062161 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.062241 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.077898 (+ 15657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.077903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.077906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.077942 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.080080 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.080802 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.080814 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.080817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.081033 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.081378 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.081383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.081385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.081638 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.081716 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.083202 (+ 1486us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.083369 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:35:03.084463 26193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.057380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:41833 (local address 127.0.0.1:47448)
0806 18:35:03.057709 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.057723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.057875 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.058271 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.058274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.058518 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.059138 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.059150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.060317 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.060320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.061656 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.061662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.062424 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.062427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.062522 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.076811 (+ 14289us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.076832 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.077660 (+ 828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.080273 (+ 2613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.080276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.080278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.080664 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.081145 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.081149 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.081151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.081255 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.081798 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.081801 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.082989 (+ 1188us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.084063 (+ 1074us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.084291 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":1122,"spinlock_wait_cycles":9856,"thread_start_us":96,"threads_started":1}
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041301, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.190@KRBTEST.COM for oryx/127.18.125.188@KRBTEST.COM
I20260806 18:35:03.095459 26000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.087486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38057 (local address 127.0.0.1:54338)
0806 18:35:03.087625 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.087641 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.087727 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.088005 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.088009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.088243 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.088341 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.088346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.089094 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.089098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.089653 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.089660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.090129 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.090133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.090231 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.090714 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.090729 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.092061 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.093875 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.093880 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.093882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.094097 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.094388 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.094392 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.094395 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.094488 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.094883 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.094886 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.095002 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.095112 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.095282 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 18:35:03.095459 26170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.087537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.18.125.188:38057)
0806 18:35:03.087659 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.087663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.087684 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:35:03.087847 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.087850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.087893 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.087987 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.088462 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.088972 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.089779 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.090014 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.090127 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.092259 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.092263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.092265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.092298 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.093746 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.094207 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.094215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.094217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.094291 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.094581 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.094586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.094588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.094771 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.094894 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.095168 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.095282 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:35:03.097078 25403 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.125.188:38057: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846"
member_type: NON_VOTER
last_known_addr {
host: "127.18.125.188"
port: 38057
}
attrs {
promote: true
}
}
dest_uuid: "630b9ddf4d484387a5769bb7c7f07d5c"
cas_config_opid_index: 5
I20260806 18:35:03.097718 25403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } }
I20260806 18:35:03.098897 26004 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2b4c9cd7083b46b3b9d9df66473a4846 (127.18.125.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.101293 26004 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.125.188:38057
I20260806 18:35:03.101501 26142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:03.101509 26004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.101613 26004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:03.102257 26008 master.cc:537] Master@127.18.125.188:38057 shutting down...
I20260806 18:35:03.103242 26142 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 FOLLOWER]: Refusing update from remote peer 630b9ddf4d484387a5769bb7c7f07d5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 18:35:03.104039 26005 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:35:03.106590 25389 consensus_peers.cc:487] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c -> Peer 2b4c9cd7083b46b3b9d9df66473a4846 (127.18.125.188:38057): Couldn't send request to peer 2b4c9cd7083b46b3b9d9df66473a4846. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:35:03.110387 26008 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:03.110750 26008 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:03.110910 26008 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: stopping tablet replica
I20260806 18:35:03.111279 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.100749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47456 (local address 127.18.125.189:41833)
0806 18:35:03.100930 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.100935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.103324 (+ 2389us) server_negotiation.cc:407] Connection header received
0806 18:35:03.103392 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.103395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.103468 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.103574 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:03.107038 (+ 3464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.107833 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.109282 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.109995 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.110139 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.110408 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.111051 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.111146 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 18:35:03.111932 26000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.100446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:41833 (local address 127.0.0.1:47456)
0806 18:35:03.103089 (+ 2643us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.103106 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.103210 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.106733 (+ 3523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.106736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.106765 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:03.106885 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.106891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.108202 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.108206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.109129 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.109141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.110587 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.110592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.110697 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.110752 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.111666 (+ 914us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.111790 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2546}
I20260806 18:35:03.112691 26057 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 LEARNER]: Refusing update from remote peer 630b9ddf4d484387a5769bb7c7f07d5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 18:35:03.113262 26005 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 18:35:03.115612 26057 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2b4c9cd7083b46b3b9d9df66473a4846 (127.18.125.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.118330 26197 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.118489 26197 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:35:03.118400 26087 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 630b9ddf4d484387a5769bb7c7f07d5c. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: NON_VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: true } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.118944 26087 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:35:03.119230 26005 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c: attempting to promote NON_VOTER c68e5972701347c3a3f2c2516c7313b5 to VOTER
I20260806 18:35:03.119735 26005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } }
I20260806 18:35:03.121377 26000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.121047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38057 (local address 127.0.0.1:54348)
0806 18:35:03.121231 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.121297 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 18:35:03.122069 26057 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 LEARNER]: Refusing update from remote peer 630b9ddf4d484387a5769bb7c7f07d5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 18:35:03.122430 26000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.122133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38057 (local address 127.0.0.1:54362)
0806 18:35:03.122292 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.122346 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260806 18:35:03.122557 25389 consensus_peers.cc:487] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c -> Peer 2b4c9cd7083b46b3b9d9df66473a4846 (127.18.125.188:38057): Couldn't send request to peer 2b4c9cd7083b46b3b9d9df66473a4846. Status: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 18:35:03.122722 26198 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 18:35:03.124637 26004 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c68e5972701347c3a3f2c2516c7313b5 (127.18.125.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.126052 26004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.126173 26004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:03.126349 26004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.126446 26004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:03.127228 26057 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c68e5972701347c3a3f2c2516c7313b5 (127.18.125.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.127990 26008 master.cc:552] Master@127.18.125.188:38057 shutdown complete.
I20260806 18:35:03.128038 26008 master_runner.cc:305] Clearing existing system tablet
I20260806 18:35:03.128393 26008 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:03.128453 26008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:03.129621 26087 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.129778 26087 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:35:03.130483 26008 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:35:03.131377 26201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:03.131601 26008 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:03.131736 26008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:35:03.131772 26008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/wal
uuid: "2b4c9cd7083b46b3b9d9df66473a4846"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "0fb2ef45b2e48796bc3795315230abb4"
server_key_iv: "3cfab1e64c3987ae7ed30ef1d239b408"
server_key_version: "encryptionkey@0"
I20260806 18:35:03.133945 26008 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:03.136101 26008 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 18:35:03.136257 26008 log.cc:1196] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 18:35:03.136583 26008 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: Deleting consensus metadata
I20260806 18:35:03.136905 26008 master_runner.cc:315] Copying system tablet from 127.18.125.190:42437
I20260806 18:35:03.138171 26008 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: tablet copy: Beginning tablet copy session from remote peer at address 127.18.125.190:42437
I20260806 18:35:03.147347 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.139422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53232 (local address 127.18.125.190:42437)
0806 18:35:03.139561 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.139566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.139586 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:35:03.139627 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.139629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.139674 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.139768 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.140419 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.140978 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.141895 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.142155 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.142327 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.143638 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.143642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.143645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.143675 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.145497 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.145989 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.145993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.145994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.146086 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.146441 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.146445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.146447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.146652 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.146765 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.147048 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.147169 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:35:03.147370 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.138830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:53232)
0806 18:35:03.139102 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.139120 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.139245 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.139819 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.139823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.140077 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.140273 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.140281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.141101 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.141106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.141697 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.141705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.142336 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.142340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.142447 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.143094 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.143108 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.143475 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.145641 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.145644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.145646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.145874 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.146225 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.146228 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.146230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.146319 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.146760 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.146763 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.146872 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.146982 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.147152 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":122,"threads_started":1}
I20260806 18:35:03.150060 25425 tablet_copy_service.cc:140] P 630b9ddf4d484387a5769bb7c7f07d5c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2b4c9cd7083b46b3b9d9df66473a4846 ({username='oryx', principal='oryx/127.18.125.188@KRBTEST.COM'} at 127.0.0.1:53232)
I20260806 18:35:03.150184 25425 tablet_copy_service.cc:161] P 630b9ddf4d484387a5769bb7c7f07d5c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2b4c9cd7083b46b3b9d9df66473a4846 at {username='oryx', principal='oryx/127.18.125.188@KRBTEST.COM'} at 127.0.0.1:53232: session id = 2b4c9cd7083b46b3b9d9df66473a4846-00000000000000000000000000000000
I20260806 18:35:03.151351 25425 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:35:03.152730 26008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 18:35:03.155303 26008 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: tablet copy: Starting download of 0 data blocks...
I20260806 18:35:03.155530 26008 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: tablet copy: Starting download of 1 WAL segments...
I20260806 18:35:03.157269 26008 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:35:03.159101 25425 tablet_copy_service.cc:342] P 630b9ddf4d484387a5769bb7c7f07d5c: Request end of tablet copy session 2b4c9cd7083b46b3b9d9df66473a4846-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.125.188@KRBTEST.COM'} at 127.0.0.1:53232
I20260806 18:35:03.159207 25425 tablet_copy_service.cc:433] P 630b9ddf4d484387a5769bb7c7f07d5c: ending tablet copy session 2b4c9cd7083b46b3b9d9df66473a4846-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2b4c9cd7083b46b3b9d9df66473a4846
I20260806 18:35:03.166054 26008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:03.167913 26208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:03.168011 26211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:03.172214 25754 heartbeater.cc:503] Master 127.18.125.190:42437 was elected leader, sending a full tablet report...
W20260806 18:35:03.172371 26209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:03.172633 26008 server_base.cc:1034] running on GCE node
I20260806 18:35:03.172827 26008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:03.173193 26008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:03.174368 26008 hybrid_clock.cc:648] HybridClock initialized: now 1786041303174354 us; error 34 us; skew 500 ppm
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:03.176003 26008 init.cc:377] Logged in from keytab as oryx/127.18.125.188@KRBTEST.COM (short username oryx)
I20260806 18:35:03.176606 26008 webserver.cc:460] Webserver started at http://127.18.125.188:33419/ using document root <none> and password file <none>
I20260806 18:35:03.176828 26008 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:03.176885 26008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:03.178270 26008 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 18:35:03.179283 26219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:03.179531 26008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 18:35:03.179633 26008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/wal
uuid: "2b4c9cd7083b46b3b9d9df66473a4846"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "0fb2ef45b2e48796bc3795315230abb4"
server_key_iv: "3cfab1e64c3987ae7ed30ef1d239b408"
server_key_version: "encryptionkey@0"
I20260806 18:35:03.179749 26008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:03.190908 26008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:03.193216 26008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:03.198626 26008 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.188:38057
I20260806 18:35:03.199239 26008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
I20260806 18:35:03.199698 26275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.188:38057 every 8 connection(s)
I20260806 18:35:03.201325 26276 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:35:03.202297 26276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: Bootstrap starting.
I20260806 18:35:03.212059 26276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:35:03.212457 26276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: Bootstrap complete.
I20260806 18:35:03.213021 26276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } }
I20260806 18:35:03.213178 26276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2b4c9cd7083b46b3b9d9df66473a4846, State: Initialized, Role: LEARNER
I20260806 18:35:03.213301 26276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } }
I20260806 18:35:03.213698 26279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.213887 26276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:03.213899 26279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:35:03.217576 26290 catalog_manager.cc:1259] Loaded cluster ID: e60b7aa24ba2482a9ef3925ff72b8032
I20260806 18:35:03.217723 26290 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: loading cluster ID for follower catalog manager: success
I20260806 18:35:03.219388 26290 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: acquiring CA information for follower catalog manager: success
I20260806 18:35:03.220770 26290 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 18:35:03.261349 26194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:03.261754 26194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:03.261832 26194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:03.268388 26194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:03.268501 26194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:03.268596 26194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:35:03.275504 26194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab
--principal=oryx/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:41241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=40867
--tserver_master_addrs=127.18.125.190:42437,127.18.125.189:41833,127.18.125.188:38057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:03.276782 26194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:03.277925 26194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:35:03.288718 26194 server_base.cc:1034] running on GCE node
W20260806 18:35:03.288658 26296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:03.288658 26295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:03.288655 26298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:03.289444 26194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:03.290225 26194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:03.291446 26194 hybrid_clock.cc:648] HybridClock initialized: now 1786041303291428 us; error 32 us; skew 500 ppm
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:03.294514 26194 init.cc:377] Logged in from keytab as oryx/127.18.125.129@KRBTEST.COM (short username oryx)
I20260806 18:35:03.295550 26194 webserver.cc:460] Webserver started at http://127.18.125.129:40867/ using document root <none> and password file <none>
I20260806 18:35:03.296249 26194 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:03.296370 26194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:03.300493 26194 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:35:03.302856 26305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:03.304303 26194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:03.304448 26194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "f443fc0a32ec415297569cab7e77e2a2"
format_stamp: "Formatted at 2026-08-06 18:35:01 on dist-test-slave-8w9v"
server_key: "241bf29e937ca9b2d2a0edc3c573eb05"
server_key_iv: "a1f1d95bad61732fae2ce5a286e276f8"
server_key_version: "encryptionkey@0"
I20260806 18:35:03.305109 26194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:03.323964 26194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:03.327526 26194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:03.327771 26194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:03.328526 26194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:03.329550 26194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:03.329624 26194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:03.329698 26194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:03.329751 26194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:03.340345 26194 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:41241
I20260806 18:35:03.340373 26424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:41241 every 8 connection(s)
I20260806 18:35:03.341619 26194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:03.343434 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 26194
I20260806 18:35:03.343927 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 25611
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.189@KRBTEST.COM
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.188@KRBTEST.COM
I20260806 18:35:03.363731 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.346776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:55693 (local address 127.18.125.190:42437)
0806 18:35:03.346956 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.346964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.346984 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:35:03.347584 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.347588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.347661 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.347770 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.350324 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.351186 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.353617 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.354029 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.354124 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.358863 (+ 4739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.358866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.358868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.358899 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.360692 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.361459 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.361467 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.361471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.361570 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.362071 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.362075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.362077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.362412 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.362510 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.363376 (+ 866us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.363526 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:35:03.364692 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:33231
--local_ip_for_outbound_sockets=127.18.125.130
--tserver_master_addrs=127.18.125.190:42437
--webserver_port=33079
--webserver_interface=127.18.125.130
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab
--principal=oryx/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.125.190:42437,127.18.125.189:41833
--tserver_master_addrs=127.18.125.190:42437,127.18.125.189:41833,127.18.125.188:38057 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:35:03.365079 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.346424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:42255 (local address 127.18.125.189:41833)
0806 18:35:03.346581 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.346587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.347969 (+ 1382us) server_negotiation.cc:407] Connection header received
0806 18:35:03.348272 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.348276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.348336 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.348408 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.350324 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.351186 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.353621 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.354029 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.354124 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.361291 (+ 7167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.361295 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.361299 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.361333 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.363098 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.363623 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.363627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.363629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.363716 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.364065 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.364069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.364070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.364331 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.364424 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.364775 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.364933 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 18:35:03.365623 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.345002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38057 (local address 127.18.125.129:32839)
0806 18:35:03.346550 (+ 1548us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.346568 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.347361 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.348184 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.348195 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.349128 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.350094 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.350140 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.352009 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.352016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.353461 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.353478 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.354546 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.354553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.354945 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.356042 (+ 1097us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.356076 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.359770 (+ 3694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.362059 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.362065 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.362070 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.362332 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.362703 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.362708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.362713 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.362806 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.363347 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.363364 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.363482 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.363605 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.364032 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1469,"mutex_wait_us":297,"spinlock_wait_cycles":91520,"thread_start_us":171,"threads_started":1}
I20260806 18:35:03.365621 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.346250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:41833 (local address 127.18.125.129:42255)
0806 18:35:03.347860 (+ 1610us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.347878 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.348048 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.348609 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.348612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.349105 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.350093 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.350136 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.351425 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.351429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.353430 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.353444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.354483 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.354487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.354947 (+ 460us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.356345 (+ 1398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.356368 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.360830 (+ 4462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.363247 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.363252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.363257 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.363500 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.363851 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.363854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.363856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.363948 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.364485 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.364489 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.364577 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.365237 (+ 660us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.365405 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1531,"mutex_wait_us":476,"spinlock_wait_cycles":16128,"thread_start_us":137,"threads_started":1}
I20260806 18:35:03.365621 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.344888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.129:55693)
0806 18:35:03.346144 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.346202 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.347329 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.348177 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.348192 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.349105 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.350093 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.350136 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.351425 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.351429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.353430 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.353444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.354483 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.354487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.354931 (+ 444us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.356026 (+ 1095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.356059 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.358594 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.360845 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.360852 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.360874 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.361320 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.361702 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.361710 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.361716 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.361940 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.362595 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.362602 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.363038 (+ 436us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.364214 (+ 1176us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.364410 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1023,"mutex_wait_us":720,"spinlock_wait_cycles":62592,"thread_start_us":172,"threads_started":1}
I20260806 18:35:03.366597 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.344457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:32839 (local address 127.18.125.188:38057)
0806 18:35:03.345556 (+ 1099us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.345580 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.346761 (+ 1181us) server_negotiation.cc:407] Connection header received
0806 18:35:03.347734 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.347741 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.347831 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.347948 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.350333 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.351834 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.353660 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.354192 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.354334 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.359981 (+ 5647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.359988 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.359994 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.360033 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.361918 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.362470 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.362476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.362481 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.362577 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.362908 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.362914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.362919 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.363195 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.363298 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.366248 (+ 2950us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.366451 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":412,"threads_started":1}
I20260806 18:35:03.367339 26425 heartbeater.cc:347] Connected to a master server at 127.18.125.188:38057
I20260806 18:35:03.367651 26425 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:03.368067 26426 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42437
I20260806 18:35:03.368175 26426 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:03.368417 26425 heartbeater.cc:511] Master 127.18.125.188:38057 requested a full tablet report, sending...
I20260806 18:35:03.368444 26427 heartbeater.cc:347] Connected to a master server at 127.18.125.189:41833
I20260806 18:35:03.368575 26427 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:03.368772 26427 heartbeater.cc:511] Master 127.18.125.189:41833 requested a full tablet report, sending...
I20260806 18:35:03.368417 26426 heartbeater.cc:511] Master 127.18.125.190:42437 requested a full tablet report, sending...
I20260806 18:35:03.370048 25403 ts_manager.cc:194] Re-registered known tserver with Master: f443fc0a32ec415297569cab7e77e2a2 (127.18.125.129:41241)
I20260806 18:35:03.370280 26047 ts_manager.cc:194] Registered new tserver with Master: f443fc0a32ec415297569cab7e77e2a2 (127.18.125.129:41241)
I20260806 18:35:03.370299 26236 ts_manager.cc:194] Registered new tserver with Master: f443fc0a32ec415297569cab7e77e2a2 (127.18.125.129:41241)
I20260806 18:35:03.370859 25403 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:55693
I20260806 18:35:03.406786 25896 heartbeater.cc:503] Master 127.18.125.190:42437 was elected leader, sending a full tablet report...
W20260806 18:35:03.489910 26433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:03.490218 26433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:03.490305 26433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:03.494652 26433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:03.494760 26433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:03.494871 26433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:35:03.500670 26433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab
--principal=oryx/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:33231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=33079
--tserver_master_addrs=127.18.125.190:42437,127.18.125.189:41833,127.18.125.188:38057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:03.501813 26433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:03.502892 26433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:03.513443 26439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:03.513443 26438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:03.513505 26441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:03.513880 26433 server_base.cc:1034] running on GCE node
I20260806 18:35:03.514277 26433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:03.515008 26433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:03.516217 26433 hybrid_clock.cc:648] HybridClock initialized: now 1786041303516203 us; error 28 us; skew 500 ppm
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:03.519040 26433 init.cc:377] Logged in from keytab as oryx/127.18.125.130@KRBTEST.COM (short username oryx)
I20260806 18:35:03.520016 26433 webserver.cc:460] Webserver started at http://127.18.125.130:33079/ using document root <none> and password file <none>
I20260806 18:35:03.520660 26433 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:03.520783 26433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:03.524969 26433 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:35:03.527297 26449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:03.529436 26433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:03.529637 26433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "dd88c02290fb40238ca877dda174f5a9"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "fd987e455ea803fe7281c4202811ad5b"
server_key_iv: "65d61a0be6b6807d9f09bdec5c05e975"
server_key_version: "encryptionkey@0"
I20260806 18:35:03.530267 26433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:03.570245 26433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:03.573566 26433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:03.573805 26433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:03.574527 26433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:03.575544 26433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:03.575625 26433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:03.575709 26433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:03.575759 26433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:03.587203 26433 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:33231
I20260806 18:35:03.587256 26577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:33231 every 8 connection(s)
I20260806 18:35:03.588390 26433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:03.594604 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 26433
I20260806 18:35:03.594795 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 25759
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.188@KRBTEST.COM
I20260806 18:35:03.603830 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:36445
--local_ip_for_outbound_sockets=127.18.125.131
--tserver_master_addrs=127.18.125.190:42437
--webserver_port=39115
--webserver_interface=127.18.125.131
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab
--principal=oryx/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:44591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.125.190:42437,127.18.125.189:41833
--tserver_master_addrs=127.18.125.190:42437,127.18.125.189:41833,127.18.125.188:38057 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.189@KRBTEST.COM
I20260806 18:35:03.607182 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.590513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:54851 (local address 127.18.125.188:38057)
0806 18:35:03.590670 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.590675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.591644 (+ 969us) server_negotiation.cc:407] Connection header received
0806 18:35:03.592838 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.592842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.592916 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.592986 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.594572 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.595424 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.598391 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.598752 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.598868 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.602205 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.602213 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.602221 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.602259 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.604758 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.605292 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.605296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.605298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.605388 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.605729 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.605732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.605734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.605940 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.606020 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.606862 (+ 842us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.607010 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:35:03.607182 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.590346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:42597 (local address 127.18.125.190:42437)
0806 18:35:03.590519 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.590524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.591945 (+ 1421us) server_negotiation.cc:407] Connection header received
0806 18:35:03.592933 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.592936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.593022 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.593099 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.594564 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.595403 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.597445 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.597798 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.597917 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.601781 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.601785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.601788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.601817 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.604370 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.605073 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.605079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.605082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.605174 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.605655 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.605662 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.605666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.605922 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.606015 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.606862 (+ 847us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.607009 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:35:03.609110 26583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.590796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.130:42597)
0806 18:35:03.591844 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.591857 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.592607 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.593209 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.593212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.593831 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.594388 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.594410 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.595924 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.595930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.597306 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.597318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.597976 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.597983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.598238 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.598977 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.599003 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.601609 (+ 2606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.604548 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.604556 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.604577 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.604952 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.605285 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.605288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.605290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.605487 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.606092 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.606100 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.606545 (+ 445us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.607419 (+ 874us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.607887 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"mutex_wait_us":156,"spinlock_wait_cycles":46720,"thread_start_us":139,"threads_started":1}
I20260806 18:35:03.609330 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.591937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:42935 (local address 127.18.125.189:41833)
0806 18:35:03.592066 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.592070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.592290 (+ 220us) server_negotiation.cc:407] Connection header received
0806 18:35:03.593150 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.593153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.593214 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.593304 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.594569 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.595402 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.601452 (+ 6050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.601907 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.602025 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.604663 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.604670 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.604676 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.604712 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.606622 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.607205 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.607211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.607216 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.607305 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.608146 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.608152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.608157 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.608422 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.608505 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.608942 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.609092 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:35:03.609422 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.590796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38057 (local address 127.18.125.130:54851)
0806 18:35:03.591415 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.591485 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.592607 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.593201 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.593211 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.593830 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.594408 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.594415 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.597492 (+ 3077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.597495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.598258 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.598267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.599016 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.599019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.599090 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.599559 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.599572 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.602021 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.604922 (+ 2901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.604925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.604926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.605176 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.605517 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.605523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.605528 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.605622 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.606092 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.606100 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.606545 (+ 445us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.607419 (+ 874us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.607939 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":364,"spinlock_wait_cycles":16384,"thread_start_us":682,"threads_started":1}
I20260806 18:35:03.610541 26579 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42437
I20260806 18:35:03.610587 26578 heartbeater.cc:347] Connected to a master server at 127.18.125.188:38057
I20260806 18:35:03.610816 26578 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:03.610816 26579 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:03.610584 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.591833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:41833 (local address 127.18.125.130:42935)
0806 18:35:03.592194 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.592207 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.592614 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.593346 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.593348 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.593752 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.594397 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.594416 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.600386 (+ 5970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.600389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.601304 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.601317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.602193 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.602199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.602314 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.602751 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.602770 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.604503 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.606807 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.606810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.606812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.607080 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.607875 (+ 795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.607881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.607886 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.607986 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.608591 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.608596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.608715 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.610113 (+ 1398us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.610312 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":48,"spinlock_wait_cycles":12544,"thread_start_us":85,"threads_started":1}
I20260806 18:35:03.611467 26579 heartbeater.cc:511] Master 127.18.125.190:42437 requested a full tablet report, sending...
I20260806 18:35:03.611482 26578 heartbeater.cc:511] Master 127.18.125.188:38057 requested a full tablet report, sending...
I20260806 18:35:03.612419 26580 heartbeater.cc:347] Connected to a master server at 127.18.125.189:41833
I20260806 18:35:03.612515 26580 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:03.612720 26580 heartbeater.cc:511] Master 127.18.125.189:41833 requested a full tablet report, sending...
I20260806 18:35:03.612739 25403 ts_manager.cc:194] Re-registered known tserver with Master: dd88c02290fb40238ca877dda174f5a9 (127.18.125.130:33231)
I20260806 18:35:03.612813 26236 ts_manager.cc:194] Registered new tserver with Master: dd88c02290fb40238ca877dda174f5a9 (127.18.125.130:33231)
I20260806 18:35:03.613564 26047 ts_manager.cc:194] Registered new tserver with Master: dd88c02290fb40238ca877dda174f5a9 (127.18.125.130:33231)
I20260806 18:35:03.613559 25403 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:42597
W20260806 18:35:03.728752 26585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:03.729172 26585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:03.729236 26585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:35:03.730621 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.725976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52058 (local address 127.18.125.188:38057)
0806 18:35:03.726390 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.726395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.726424 (+ 29us) server_negotiation.cc:407] Connection header received
0806 18:35:03.726498 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.726501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.726590 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.726748 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:03.727210 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.727815 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.729520 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.730018 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.730144 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.730266 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.730435 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.730508 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264}
I20260806 18:35:03.730641 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.725819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38057 (local address 127.0.0.1:52058)
0806 18:35:03.726237 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.726253 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.726371 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.726798 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.726801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.726827 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:03.726967 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.726974 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.727981 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.727985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.729347 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.729364 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.730169 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.730173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.730274 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.730305 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.730389 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.730484 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":100,"threads_started":1}
I20260806 18:35:03.731385 26248 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 18:35:03.731920 26591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 630b9ddf4d484387a5769bb7c7f07d5c. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.732160 26591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: This master's current role is: LEARNER
W20260806 18:35:03.734934 26585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:03.735277 26585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:03.735381 26585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:35:03.743211 26585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:44591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab
--principal=oryx/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:36445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=39115
--tserver_master_addrs=127.18.125.190:42437,127.18.125.189:41833,127.18.125.188:38057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:03.745015 26585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:03.746215 26585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:03.759442 26594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:03.759464 26595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:03.759507 26585 server_base.cc:1034] running on GCE node
W20260806 18:35:03.759442 26597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:03.760221 26585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:03.760938 26585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:03.762375 26585 hybrid_clock.cc:648] HybridClock initialized: now 1786041303762337 us; error 39 us; skew 500 ppm
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:03.766196 26585 init.cc:377] Logged in from keytab as oryx/127.18.125.131@KRBTEST.COM (short username oryx)
I20260806 18:35:03.767400 26585 webserver.cc:460] Webserver started at http://127.18.125.131:39115/ using document root <none> and password file <none>
I20260806 18:35:03.768061 26585 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:03.768245 26585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:03.772897 26585 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:35:03.775524 26605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:03.777882 26585 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:35:03.778100 26585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "fe73b370c34842228aa10e9f0a150438"
format_stamp: "Formatted at 2026-08-06 18:35:02 on dist-test-slave-8w9v"
server_key: "3996563f16fcb46002eeaa8daccfec12"
server_key_iv: "170298862b7cfa7844b8803c4192b20b"
server_key_version: "encryptionkey@0"
I20260806 18:35:03.779345 26585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:03.798054 26585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:03.801880 26585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:03.802115 26585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:03.802835 26585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:03.803810 26585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:03.803889 26585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:03.803962 26585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:03.803997 26585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:03.818897 26585 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:36445
I20260806 18:35:03.819139 26729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:36445 every 8 connection(s)
I20260806 18:35:03.820335 26585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:03.823072 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 26585
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.189@KRBTEST.COM
I20260806 18:35:03.838449 26742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.825359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47844 (local address 127.18.125.190:42437)
0806 18:35:03.825935 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.825941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.826706 (+ 765us) server_negotiation.cc:407] Connection header received
0806 18:35:03.826859 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.826862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.826920 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.827013 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.828044 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.828925 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.830164 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.830528 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.830618 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.832753 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.832760 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.832765 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.832799 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.835637 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.836264 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.836270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.836273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.836373 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.836737 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.836741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.836743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.837001 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.837134 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.837956 (+ 822us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.838122 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":92,"threads_started":1}
I20260806 18:35:03.842898 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.822634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:41431 (local address 127.18.125.190:42437)
0806 18:35:03.822820 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.822826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.824534 (+ 1708us) server_negotiation.cc:407] Connection header received
0806 18:35:03.827256 (+ 2722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.827260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.827312 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.827366 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.829768 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.830604 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.832653 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.832992 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.833073 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.837618 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.837624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.837630 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.837663 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.839637 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.840697 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.840702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.840704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.840816 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.841327 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.841331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.841333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.841603 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.841704 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.842553 (+ 849us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.842708 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:35:03.844959 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.825893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55950 (local address 127.18.125.189:41833)
0806 18:35:03.826967 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.826971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.828871 (+ 1900us) server_negotiation.cc:407] Connection header received
0806 18:35:03.828918 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.828921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.828969 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.829067 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.834270 (+ 5203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.835293 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.836525 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.836882 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.837029 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.839765 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.839769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.839774 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.839837 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.842049 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.842778 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.842782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.842783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.842888 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.843214 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.843218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.843219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.843470 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.843622 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.844322 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.844497 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004,"thread_start_us":90,"threads_started":1}
I20260806 18:35:03.845432 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.823114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.131:41431)
0806 18:35:03.824346 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.824410 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.827080 (+ 2670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.827821 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.827832 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.829503 (+ 1671us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.829654 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.829663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.830777 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.830781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.832517 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.832528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.833319 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.833324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.833720 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.834415 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.834445 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.837377 (+ 2932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.840019 (+ 2642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.840027 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.840051 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.840565 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.840953 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.840960 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.840962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.841186 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.841796 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.841805 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.842193 (+ 388us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.843360 (+ 1167us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.843784 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"mutex_wait_us":263,"thread_start_us":700,"threads_started":1}
Aug 06 18:35:03 dist-test-slave-8w9v krb5kdc[25353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041303, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.188@KRBTEST.COM
I20260806 18:35:03.847523 26731 heartbeater.cc:347] Connected to a master server at 127.18.125.190:42437
I20260806 18:35:03.847813 26731 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:03.843591 25403 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:35:03.848609 26731 heartbeater.cc:511] Master 127.18.125.190:42437 requested a full tablet report, sending...
W20260806 18:35:03.849875 25403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:35:03.850621 25402 ts_manager.cc:194] Re-registered known tserver with Master: fe73b370c34842228aa10e9f0a150438 (127.18.125.131:36445)
I20260806 18:35:03.850929 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.845034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52062 (local address 127.18.125.188:38057)
0806 18:35:03.845402 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.845407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.845788 (+ 381us) server_negotiation.cc:407] Connection header received
0806 18:35:03.845881 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.845884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.846040 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.846534 (+ 494us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:03.846667 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.847412 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.848664 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.849010 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.849460 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.849559 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:03.850105 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:03.850656 (+ 551us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.850698 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.850766 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":134,"threads_started":1}
I20260806 18:35:03.851492 25402 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:41431
I20260806 18:35:03.853036 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.826125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:48715 (local address 127.18.125.188:38057)
0806 18:35:03.826282 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.826287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.837539 (+ 11252us) server_negotiation.cc:407] Connection header received
0806 18:35:03.840002 (+ 2463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.840007 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.840144 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.840259 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.841336 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.842145 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.843530 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.843892 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.843988 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.847752 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.847756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.847759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.847793 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.849912 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.850723 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.850728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.850730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.850842 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.851507 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.851514 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.851518 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.851776 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.851878 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.852708 (+ 830us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.852872 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:35:03.853415 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.825267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38057 (local address 127.18.125.131:48715)
0806 18:35:03.837402 (+ 12135us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.837420 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.839854 (+ 2434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.840739 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.840744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.840981 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.841158 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.841167 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.842325 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.842328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.843203 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.843385 (+ 182us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.844249 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.844253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.844389 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.845692 (+ 1303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.845726 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.847535 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.850083 (+ 2548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.850090 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.850094 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.850397 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.850994 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.850997 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.850999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.851372 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.852351 (+ 979us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.852354 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.852477 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.852608 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.852809 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12018,"thread_start_us":147,"threads_started":1}
I20260806 18:35:03.853868 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.824892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:41833 (local address 127.18.125.131:33317)
0806 18:35:03.825681 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.825699 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.826882 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.827821 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.827832 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.828483 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:03.829288 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.829318 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.830777 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.830781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.832453 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.832468 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.833319 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.833324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.833861 (+ 537us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.834695 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:03.834711 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:03.837834 (+ 3123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:03.841747 (+ 3913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.841751 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.841754 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.852061 (+ 10307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.852654 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:03.852658 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:03.852660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:03.852762 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:03.853328 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:03.853330 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:03.853415 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.853520 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.853690 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":120,"thread_start_us":95,"threads_started":1}
I20260806 18:35:03.853881 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.825018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:33317 (local address 127.18.125.189:41833)
0806 18:35:03.825139 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.825145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.825775 (+ 630us) server_negotiation.cc:407] Connection header received
0806 18:35:03.827256 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.827259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.827312 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.827366 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:03.829837 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.830619 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.832617 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.832993 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.833073 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.838017 (+ 4944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:03.838021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:03.838024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:03.838051 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:03.839875 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.852303 (+ 12428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.852308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.852309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.852445 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:03.852906 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:03.852910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:03.852912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:03.853158 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:03.853264 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.853595 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.853730 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 18:35:03.854354 26730 heartbeater.cc:347] Connected to a master server at 127.18.125.188:38057
I20260806 18:35:03.854449 26730 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:03.854668 26730 heartbeater.cc:511] Master 127.18.125.188:38057 requested a full tablet report, sending...
I20260806 18:35:03.854820 26732 heartbeater.cc:347] Connected to a master server at 127.18.125.189:41833
I20260806 18:35:03.854916 26732 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:03.855083 26732 heartbeater.cc:511] Master 127.18.125.189:41833 requested a full tablet report, sending...
I20260806 18:35:03.855491 26236 ts_manager.cc:194] Registered new tserver with Master: fe73b370c34842228aa10e9f0a150438 (127.18.125.131:36445)
I20260806 18:35:03.855994 26047 ts_manager.cc:194] Registered new tserver with Master: fe73b370c34842228aa10e9f0a150438 (127.18.125.131:36445)
I20260806 18:35:03.856447 26198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: NON_VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: true } } }
I20260806 18:35:03.856585 26198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:03.860891 26198 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c: attempting to promote NON_VOTER 2b4c9cd7083b46b3b9d9df66473a4846 to VOTER
I20260806 18:35:03.861320 26198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } }
I20260806 18:35:03.862649 26057 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Refusing update from remote peer 630b9ddf4d484387a5769bb7c7f07d5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 18:35:03.863086 26749 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 18:35:03.863554 26248 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 LEARNER]: Refusing update from remote peer 630b9ddf4d484387a5769bb7c7f07d5c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 18:35:03.864079 26749 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 18:35:03.865015 26750 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2b4c9cd7083b46b3b9d9df66473a4846 (127.18.125.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:03.865568 26057 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2b4c9cd7083b46b3b9d9df66473a4846 (127.18.125.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:03.866791 26750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:03.866767 26198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:03.866990 26198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:03.866853 26248 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2b4c9cd7083b46b3b9d9df66473a4846 (127.18.125.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:03.867154 26198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:03.867259 26198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:03.866938 26750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:03.868356 26759 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:03.868592 26759 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:35:03.869132 26591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:03.869253 26591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:35:03.877116 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.871377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:36445 (local address 127.0.0.1:57882)
0806 18:35:03.871617 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.871631 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.871717 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.872585 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.872588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.872607 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:03.872724 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.872729 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.874209 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.874212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.875418 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.875429 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.876605 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.876612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.876728 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.876757 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.876844 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.876949 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":3,"thread_start_us":108,"threads_started":1}
I20260806 18:35:03.877146 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.871643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57882 (local address 127.18.125.131:36445)
0806 18:35:03.871984 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.871999 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.872025 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:35:03.872111 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.872164 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.872418 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.872602 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:03.872874 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.874072 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.875558 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.876462 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.876607 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.876760 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.876886 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.876990 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":140,"threads_started":1}
I20260806 18:35:03.877377 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.872211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33894 (local address 127.18.125.129:41241)
0806 18:35:03.872470 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.872475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.872494 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:35:03.872587 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.872592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.872776 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.873202 (+ 426us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:03.873339 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.874603 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.875887 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.876557 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.876943 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.877069 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.877119 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.877227 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":89,"threads_started":1}
I20260806 18:35:03.877796 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.871809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:41241 (local address 127.0.0.1:33894)
0806 18:35:03.872095 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.872111 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.872231 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.872930 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.872935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.872956 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:03.873052 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.873058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.874740 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.874743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.875762 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.875770 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.876689 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.876695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.876768 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.876794 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.876842 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.877671 (+ 829us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":726,"thread_start_us":150,"threads_started":1}
I20260806 18:35:03.878160 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.871047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:33231 (local address 127.0.0.1:47256)
0806 18:35:03.871166 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.871180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.871299 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.873374 (+ 2075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.873377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.873396 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:03.873481 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.873487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.875335 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.875340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.876496 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.876509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.877750 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.877754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.877842 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.877874 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.877959 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.878046 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 18:35:03.878391 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.872582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.18.125.130:33231)
0806 18:35:03.872903 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.872915 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.872935 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:35:03.873015 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.873022 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.873208 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.873804 (+ 596us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:03.873983 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.875212 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.876640 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.877652 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.877769 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.877888 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.878127 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.878237 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":113,"threads_started":1}
I20260806 18:35:03.881233 26659 tablet_service.cc:1467] Processing CreateTablet for tablet d654fd3176834c1981cb164792efdfc2 (DEFAULT_TABLE table=test-table [id=1c7bd2122def4551a74080cf966d5837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:03.881582 26504 tablet_service.cc:1467] Processing CreateTablet for tablet d654fd3176834c1981cb164792efdfc2 (DEFAULT_TABLE table=test-table [id=1c7bd2122def4551a74080cf966d5837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:03.881580 26357 tablet_service.cc:1467] Processing CreateTablet for tablet d654fd3176834c1981cb164792efdfc2 (DEFAULT_TABLE table=test-table [id=1c7bd2122def4551a74080cf966d5837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:03.882344 26659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d654fd3176834c1981cb164792efdfc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:03.882596 26504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d654fd3176834c1981cb164792efdfc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:03.882596 26357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d654fd3176834c1981cb164792efdfc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:03.890575 26767 tablet_bootstrap.cc:492] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2: Bootstrap starting.
I20260806 18:35:03.891436 26768 tablet_bootstrap.cc:492] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9: Bootstrap starting.
I20260806 18:35:03.891729 26766 tablet_bootstrap.cc:492] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438: Bootstrap starting.
I20260806 18:35:03.893337 26768 tablet_bootstrap.cc:654] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:03.893450 26767 tablet_bootstrap.cc:654] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:03.893816 26766 tablet_bootstrap.cc:654] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:03.894219 26768 log.cc:824] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:03.894332 26767 log.cc:824] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:03.894578 26766 log.cc:824] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:03.896600 26768 tablet_bootstrap.cc:492] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9: No bootstrap required, opened a new log
I20260806 18:35:03.896761 26767 tablet_bootstrap.cc:492] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2: No bootstrap required, opened a new log
I20260806 18:35:03.896845 26768 ts_tablet_manager.cc:1397] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260806 18:35:03.896971 26767 ts_tablet_manager.cc:1397] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260806 18:35:03.899909 26767 raft_consensus.cc:348] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:03.900209 26767 raft_consensus.cc:374] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:03.900274 26767 raft_consensus.cc:729] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f443fc0a32ec415297569cab7e77e2a2, State: Initialized, Role: FOLLOWER
I20260806 18:35:03.900283 26768 raft_consensus.cc:348] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:03.900570 26768 raft_consensus.cc:374] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:03.900640 26768 raft_consensus.cc:729] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd88c02290fb40238ca877dda174f5a9, State: Initialized, Role: FOLLOWER
I20260806 18:35:03.900802 26767 consensus_queue.cc:260] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:03.901078 26768 consensus_queue.cc:260] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:03.901858 26767 ts_tablet_manager.cc:1428] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260806 18:35:03.902024 26766 tablet_bootstrap.cc:492] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438: No bootstrap required, opened a new log
I20260806 18:35:03.902029 26768 ts_tablet_manager.cc:1428] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:35:03.902154 26426 heartbeater.cc:503] Master 127.18.125.190:42437 was elected leader, sending a full tablet report...
I20260806 18:35:03.902439 26766 ts_tablet_manager.cc:1397] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260806 18:35:03.904785 26579 heartbeater.cc:503] Master 127.18.125.190:42437 was elected leader, sending a full tablet report...
I20260806 18:35:03.907428 26766 raft_consensus.cc:348] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:03.907744 26766 raft_consensus.cc:374] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:03.907815 26766 raft_consensus.cc:729] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe73b370c34842228aa10e9f0a150438, State: Initialized, Role: FOLLOWER
I20260806 18:35:03.908314 26766 consensus_queue.cc:260] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:03.909441 26766 ts_tablet_manager.cc:1428] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:35:03.909807 26731 heartbeater.cc:503] Master 127.18.125.190:42437 was elected leader, sending a full tablet report...
I20260806 18:35:03.926652 26774 raft_consensus.cc:482] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:35:03.926892 26774 raft_consensus.cc:504] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:03.928048 26774 leader_election.cc:288] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd88c02290fb40238ca877dda174f5a9 (127.18.125.130:33231), f443fc0a32ec415297569cab7e77e2a2 (127.18.125.129:41241)
I20260806 18:35:03.932052 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.928890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:43865 (local address 127.18.125.129:41241)
0806 18:35:03.929054 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.929060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.929075 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:35:03.929134 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.929136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.929173 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.929260 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:03.929525 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.930036 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.931042 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.931451 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.931581 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.931632 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.931837 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.931920 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:35:03.932060 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.928504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:55965 (local address 127.18.125.130:33231)
0806 18:35:03.928664 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.928669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.928685 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:35:03.928867 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.928870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.928924 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.928997 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:03.929293 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.929857 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.930939 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.931353 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.931487 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.931548 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.931740 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.931929 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:35:03.932083 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.928417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:33231 (local address 127.18.125.131:55965)
0806 18:35:03.928591 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.928610 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.928717 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.929012 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.929015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.929032 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:03.929166 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.929172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.929984 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.929989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.930787 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.930796 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.931485 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.931489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.931606 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.931627 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.931720 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.931803 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 18:35:03.932067 26740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.928742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:41241 (local address 127.18.125.131:43865)
0806 18:35:03.928874 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:03.928888 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:03.928997 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:03.929273 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:03.929276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:03.929293 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:03.929411 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.929416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.930156 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.930159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.930912 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:03.930920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.931576 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.931580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:03.931695 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.931714 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:35:03.931816 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 18:35:03.931893 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 18:35:03.933055 26379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d654fd3176834c1981cb164792efdfc2" candidate_uuid: "fe73b370c34842228aa10e9f0a150438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f443fc0a32ec415297569cab7e77e2a2" is_pre_election: true
I20260806 18:35:03.933059 26526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d654fd3176834c1981cb164792efdfc2" candidate_uuid: "fe73b370c34842228aa10e9f0a150438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd88c02290fb40238ca877dda174f5a9" is_pre_election: true
I20260806 18:35:03.933404 26526 raft_consensus.cc:2454] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe73b370c34842228aa10e9f0a150438 in term 0.
I20260806 18:35:03.933395 26379 raft_consensus.cc:2454] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe73b370c34842228aa10e9f0a150438 in term 0.
I20260806 18:35:03.934101 26608 leader_election.cc:302] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f443fc0a32ec415297569cab7e77e2a2, fe73b370c34842228aa10e9f0a150438; no voters:
I20260806 18:35:03.934537 26774 raft_consensus.cc:2781] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:35:03.934653 26774 raft_consensus.cc:482] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:35:03.934716 26774 raft_consensus.cc:3037] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:03.936964 26774 raft_consensus.cc:504] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:03.937427 26774 leader_election.cc:288] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [CANDIDATE]: Term 1 election: Requested vote from peers dd88c02290fb40238ca877dda174f5a9 (127.18.125.130:33231), f443fc0a32ec415297569cab7e77e2a2 (127.18.125.129:41241)
I20260806 18:35:03.938087 26379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d654fd3176834c1981cb164792efdfc2" candidate_uuid: "fe73b370c34842228aa10e9f0a150438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f443fc0a32ec415297569cab7e77e2a2"
I20260806 18:35:03.938243 26379 raft_consensus.cc:3037] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:03.938769 26526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d654fd3176834c1981cb164792efdfc2" candidate_uuid: "fe73b370c34842228aa10e9f0a150438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd88c02290fb40238ca877dda174f5a9"
I20260806 18:35:03.938936 26526 raft_consensus.cc:3037] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:03.939658 26379 raft_consensus.cc:2454] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe73b370c34842228aa10e9f0a150438 in term 1.
I20260806 18:35:03.939970 26608 leader_election.cc:302] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f443fc0a32ec415297569cab7e77e2a2, fe73b370c34842228aa10e9f0a150438; no voters:
I20260806 18:35:03.940186 26774 raft_consensus.cc:2781] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:03.940358 26526 raft_consensus.cc:2454] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe73b370c34842228aa10e9f0a150438 in term 1.
I20260806 18:35:03.940466 26774 raft_consensus.cc:686] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 1 LEADER]: Becoming Leader. State: Replica: fe73b370c34842228aa10e9f0a150438, State: Running, Role: LEADER
I20260806 18:35:03.940932 26774 consensus_queue.cc:237] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:03.944521 25403 catalog_manager.cc:5495] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe73b370c34842228aa10e9f0a150438 (127.18.125.131). New cstate: current_term: 1 leader_uuid: "fe73b370c34842228aa10e9f0a150438" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } health_report { overall_health: UNKNOWN } } }
I20260806 18:35:03.963708 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:03.959213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57890 (local address 127.18.125.131:36445)
0806 18:35:03.959387 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:03.959391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:03.959413 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:35:03.959546 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:03.959549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:03.959769 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:03.959916 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:03.960286 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.960872 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.961746 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:03.962156 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:03.962308 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:03.962708 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:03.963207 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:03.963316 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:03.963486 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 18:35:03.963567 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:35:03.970618 26379 raft_consensus.cc:1261] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2 [term 1 FOLLOWER]: Refusing update from remote peer fe73b370c34842228aa10e9f0a150438: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:03.970618 26526 raft_consensus.cc:1261] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9 [term 1 FOLLOWER]: Refusing update from remote peer fe73b370c34842228aa10e9f0a150438: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:03.971544 26774 consensus_queue.cc:1035] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:03.971830 26775 consensus_queue.cc:1035] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:03.982029 26778 mvcc.cc:205] Tried to move back new op lower bound from 7315625181055352832 to 7315625180945154048. Current Snapshot: MvccSnapshot[applied={T|T < 7315625181055352832}]
I20260806 18:35:03.993834 25403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:03.994031 25403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:03.998462 25403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 630b9ddf4d484387a5769bb7c7f07d5c: Sending DeleteTablet for 3 replicas of tablet d654fd3176834c1981cb164792efdfc2
I20260806 18:35:03.999791 26504 tablet_service.cc:1514] Processing DeleteTablet for tablet d654fd3176834c1981cb164792efdfc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:03 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:47256
I20260806 18:35:04.000432 26787 tablet_replica.cc:330] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9: stopping tablet replica
I20260806 18:35:04.000406 26357 tablet_service.cc:1514] Processing DeleteTablet for tablet d654fd3176834c1981cb164792efdfc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:03 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:33894
I20260806 18:35:04.000864 26787 raft_consensus.cc:2229] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:04.000941 26788 tablet_replica.cc:330] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2: stopping tablet replica
I20260806 18:35:04.001327 26787 raft_consensus.cc:2258] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:04.001376 26788 raft_consensus.cc:2229] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:04.001938 26788 raft_consensus.cc:2258] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:04.003420 26787 ts_tablet_manager.cc:1905] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:04.004043 26788 ts_tablet_manager.cc:1905] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:04.004208 26659 tablet_service.cc:1514] Processing DeleteTablet for tablet d654fd3176834c1981cb164792efdfc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:03 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:57882
I20260806 18:35:04.005336 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 25368
I20260806 18:35:04.008198 26787 ts_tablet_manager.cc:1918] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:04.008343 26787 log.cc:1196] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/d654fd3176834c1981cb164792efdfc2
I20260806 18:35:04.008745 26787 ts_tablet_manager.cc:1939] T d654fd3176834c1981cb164792efdfc2 P dd88c02290fb40238ca877dda174f5a9: Deleting consensus metadata
I20260806 18:35:04.009946 26788 ts_tablet_manager.cc:1918] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:04.010164 26788 log.cc:1196] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-0/wal/wals/d654fd3176834c1981cb164792efdfc2
I20260806 18:35:04.010638 26788 ts_tablet_manager.cc:1939] T d654fd3176834c1981cb164792efdfc2 P f443fc0a32ec415297569cab7e77e2a2: Deleting consensus metadata
I20260806 18:35:04.013021 26791 tablet_replica.cc:330] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438: stopping tablet replica
I20260806 18:35:04.013520 26791 raft_consensus.cc:2229] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:35:04.014109 26791 raft_consensus.cc:2258] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:04.017293 26791 ts_tablet_manager.cc:1905] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:35:04.017490 26308 connection.cc:535] server connection from 127.0.0.1:33894 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 18:35:04.018893 26451 connection.cc:535] server connection from 127.0.0.1:47256 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:35:04.023804 26791 ts_tablet_manager.cc:1918] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:04.024153 26791 log.cc:1196] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/d654fd3176834c1981cb164792efdfc2
I20260806 18:35:04.024700 26791 ts_tablet_manager.cc:1939] T d654fd3176834c1981cb164792efdfc2 P fe73b370c34842228aa10e9f0a150438: Deleting consensus metadata
W20260806 18:35:04.026250 26607 connection.cc:441] server connection from 127.0.0.1:57882 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57882 (request call id 1) could send its response
I20260806 18:35:04.027010 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:04.004822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.18.125.189:41833)
0806 18:35:04.005107 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:04.005112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:04.012600 (+ 7488us) server_negotiation.cc:407] Connection header received
0806 18:35:04.012799 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:04.012803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:04.012870 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:04.012954 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:04.014298 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.015486 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.017437 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.017786 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.017890 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:04.021056 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:04.021062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:04.021065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:04.021119 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:04.024289 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:04.025062 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:04.025066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:04.025068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:04.025174 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:04.025680 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:04.025684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:04.025686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:04.025967 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:04.026100 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:04.026635 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 18:35:04.026802 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260806 18:35:04.039362 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:04.025188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52082 (local address 127.18.125.188:38057)
0806 18:35:04.025377 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:04.025381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:04.026950 (+ 1569us) server_negotiation.cc:407] Connection header received
0806 18:35:04.027326 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:04.027331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:04.027397 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:04.027460 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:04.028643 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.029521 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.032290 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.032669 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.032744 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:04.034687 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:04.034691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:04.034694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:04.034726 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:04.037067 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:04.037597 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:04.037602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:04.037604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:04.037758 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:04.038107 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:04.038111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:04.038113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:04.038371 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:04.038487 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:04.038952 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 18:35:04.039109 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 18:35:04.040005 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:04.003735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52076 (local address 127.18.125.188:38057)
0806 18:35:04.003878 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:04.003885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:04.028106 (+ 24221us) server_negotiation.cc:407] Connection header received
0806 18:35:04.028871 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:04.028878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:04.028954 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:04.029015 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:04.029821 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.030422 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.031662 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.032095 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.032226 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:04.034563 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:04.034571 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:04.034576 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:04.034613 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:04.037502 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:04.038366 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:04.038371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:04.038374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:04.038484 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:04.038858 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:04.038865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:04.038869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:04.039128 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:04.039249 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:04.039675 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 18:35:04.039819 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:35:04.042276 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:04.026324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.18.125.189:41833)
0806 18:35:04.026460 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:04.026464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:04.031190 (+ 4726us) server_negotiation.cc:407] Connection header received
0806 18:35:04.031324 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:04.031327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:04.031390 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:04.031489 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:04.032522 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.033373 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.034646 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:04.035050 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:04.035161 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:04.037332 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:04.037338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:04.037342 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:04.037382 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:04.039371 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:04.040711 (+ 1340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:04.040715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:04.040717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:04.040822 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:04.041181 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:04.041187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:04.041189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:04.041453 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:04.041573 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:04.041947 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 18:35:04.042106 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:35:04.988759 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:04.988219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.130:33109)
0806 18:35:04.988529 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:04.988638 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":96,"threads_started":1}
W20260806 18:35:04.989300 26579 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:42437 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
I20260806 18:35:04.990425 26801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:04.989907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.129:54723)
0806 18:35:04.990270 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:04.990347 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":68,"threads_started":1}
I20260806 18:35:04.990679 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:04.990229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.131:50997)
0806 18:35:04.990468 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:04.990577 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
W20260806 18:35:04.990716 26426 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:42437 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
W20260806 18:35:04.990922 26731 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:42437 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
I20260806 18:35:05.497768 26803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 630b9ddf4d484387a5769bb7c7f07d5c)
I20260806 18:35:05.498024 26803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } }
I20260806 18:35:05.499192 26804 raft_consensus.cc:482] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 630b9ddf4d484387a5769bb7c7f07d5c)
I20260806 18:35:05.499280 26803 leader_election.cc:288] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437), c68e5972701347c3a3f2c2516c7313b5 (127.18.125.189:41833)
I20260806 18:35:05.499419 26804 raft_consensus.cc:504] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } }
I20260806 18:35:05.499969 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.499427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48270)
0806 18:35:05.499752 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.499854 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":81,"threads_started":1}
W20260806 18:35:05.500224 26220 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
I20260806 18:35:05.500506 26804 leader_election.cc:288] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437), 2b4c9cd7083b46b3b9d9df66473a4846 (127.18.125.188:38057)
I20260806 18:35:05.501340 26806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.500962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48292)
0806 18:35:05.501177 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.501248 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260806 18:35:05.501472 26220 leader_election.cc:334] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437): Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
I20260806 18:35:05.502064 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.501169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48282)
0806 18:35:05.501846 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.501958 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":84,"threads_started":1}
W20260806 18:35:05.502269 26029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
I20260806 18:35:05.503280 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.503020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48302)
0806 18:35:05.503152 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.503210 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 18:35:05.503397 26029 leader_election.cc:334] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437): Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
I20260806 18:35:05.506309 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.499873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:41833 (local address 127.0.0.1:55982)
0806 18:35:05.500166 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.500181 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:05.500317 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:05.501068 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:05.501073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:05.501100 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:05.501249 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:05.501256 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.502474 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.502479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:05.504969 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:05.504981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.505909 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.505912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:05.505995 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:05.506020 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:35:05.506091 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:35:05.506158 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260806 18:35:05.506397 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.500283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.18.125.189:41833)
0806 18:35:05.500795 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:05.500800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:05.500816 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:35:05.500866 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:05.500871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:05.500922 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:05.501353 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:05.501486 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.502318 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.505126 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.505771 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.505922 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:05.505987 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:05.506133 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 18:35:05.506225 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":88,"threads_started":1}
I20260806 18:35:05.506757 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.500689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38057 (local address 127.0.0.1:52086)
0806 18:35:05.501067 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.501079 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:05.501149 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:05.501764 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:05.501768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:05.501789 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:05.501873 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:05.501879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.503390 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.503394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:05.504921 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:05.504934 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.506055 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.506066 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:05.506191 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:05.506225 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 18:35:05.506312 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 18:35:05.506625 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":72,"threads_started":1}
I20260806 18:35:05.506867 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.500915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52086 (local address 127.18.125.188:38057)
0806 18:35:05.501486 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:05.501498 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:05.501517 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:35:05.501569 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:05.501573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:05.501624 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:05.501739 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:05.502010 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.503246 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.505075 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:05.505793 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:05.505922 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:05.505987 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:05.506608 (+ 621us) server_negotiation.cc:299] Negotiation successful
0806 18:35:05.506710 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":123,"threads_started":1}
I20260806 18:35:05.506867 26057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c68e5972701347c3a3f2c2516c7313b5" is_pre_election: true
I20260806 18:35:05.507121 26057 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b4c9cd7083b46b3b9d9df66473a4846 in term 1.
I20260806 18:35:05.507586 26220 leader_election.cc:302] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b4c9cd7083b46b3b9d9df66473a4846, c68e5972701347c3a3f2c2516c7313b5; no voters: 630b9ddf4d484387a5769bb7c7f07d5c
I20260806 18:35:05.507614 26248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c68e5972701347c3a3f2c2516c7313b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" is_pre_election: true
I20260806 18:35:05.507793 26248 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c68e5972701347c3a3f2c2516c7313b5 in term 1.
I20260806 18:35:05.507896 26803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 18:35:05.507972 26803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 630b9ddf4d484387a5769bb7c7f07d5c)
I20260806 18:35:05.508047 26803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:35:05.508891 26031 leader_election.cc:302] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b4c9cd7083b46b3b9d9df66473a4846, c68e5972701347c3a3f2c2516c7313b5; no voters: 630b9ddf4d484387a5769bb7c7f07d5c
I20260806 18:35:05.509150 26804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 18:35:05.509244 26804 raft_consensus.cc:482] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 630b9ddf4d484387a5769bb7c7f07d5c)
I20260806 18:35:05.509310 26804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:35:05.509768 26803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } }
I20260806 18:35:05.510242 26803 leader_election.cc:288] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [CANDIDATE]: Term 2 election: Requested vote from peers 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437), c68e5972701347c3a3f2c2516c7313b5 (127.18.125.189:41833)
I20260806 18:35:05.510828 26804 raft_consensus.cc:504] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } }
I20260806 18:35:05.510912 26057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c68e5972701347c3a3f2c2516c7313b5"
I20260806 18:35:05.511022 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.510468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48322)
0806 18:35:05.510861 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.510934 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296}
I20260806 18:35:05.511102 26057 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2b4c9cd7083b46b3b9d9df66473a4846 in current term 2: Already voted for candidate c68e5972701347c3a3f2c2516c7313b5 in this term.
I20260806 18:35:05.511358 26804 leader_election.cc:288] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 2 election: Requested vote from peers 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437), 2b4c9cd7083b46b3b9d9df66473a4846 (127.18.125.188:38057)
I20260806 18:35:05.511758 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.511491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48332)
0806 18:35:05.511617 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.511677 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 18:35:05.511971 26248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c68e5972701347c3a3f2c2516c7313b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b4c9cd7083b46b3b9d9df66473a4846"
I20260806 18:35:05.512215 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.511923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48338)
0806 18:35:05.512060 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.512111 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 18:35:05.512238 26248 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c68e5972701347c3a3f2c2516c7313b5 in current term 2: Already voted for candidate 2b4c9cd7083b46b3b9d9df66473a4846 in this term.
W20260806 18:35:05.512310 26220 leader_election.cc:334] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437): Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
I20260806 18:35:05.512408 26220 leader_election.cc:302] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b4c9cd7083b46b3b9d9df66473a4846; no voters: 630b9ddf4d484387a5769bb7c7f07d5c, c68e5972701347c3a3f2c2516c7313b5
I20260806 18:35:05.512678 26803 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 18:35:05.513142 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.512932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48342)
0806 18:35:05.513029 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.513084 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 18:35:05.513281 26029 leader_election.cc:334] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437): Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
I20260806 18:35:05.513381 26029 leader_election.cc:302] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c68e5972701347c3a3f2c2516c7313b5; no voters: 2b4c9cd7083b46b3b9d9df66473a4846, 630b9ddf4d484387a5769bb7c7f07d5c
I20260806 18:35:05.513562 26804 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 18:35:05.990695 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.989946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.130:42023)
0806 18:35:05.990489 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.990569 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":135,"threads_started":1}
I20260806 18:35:05.991710 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.991296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.129:33167)
0806 18:35:05.991553 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.991626 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":107,"threads_started":1}
I20260806 18:35:05.992288 26815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:05.991468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.131:55543)
0806 18:35:05.991919 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:05.991984 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":85,"threads_started":1}
I20260806 18:35:06.992045 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.991480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.130:45615)
0806 18:35:06.991824 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:06.991901 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":79,"threads_started":1}
I20260806 18:35:06.993007 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.992523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.129:60639)
0806 18:35:06.992837 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:06.992900 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":112,"threads_started":1}
I20260806 18:35:06.993494 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:06.993112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.131:43931)
0806 18:35:06.993362 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:06.993419 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":111,"threads_started":1}
I20260806 18:35:07.273757 26819 raft_consensus.cc:482] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:35:07.273898 26819 raft_consensus.cc:504] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } }
I20260806 18:35:07.274344 26819 leader_election.cc:288] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437), 2b4c9cd7083b46b3b9d9df66473a4846 (127.18.125.188:38057)
I20260806 18:35:07.274791 26248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c68e5972701347c3a3f2c2516c7313b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" is_pre_election: true
I20260806 18:35:07.274883 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.274485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48570)
0806 18:35:07.274738 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.274792 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":89,"threads_started":1}
I20260806 18:35:07.274981 26248 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c68e5972701347c3a3f2c2516c7313b5 in term 2.
I20260806 18:35:07.275369 26031 leader_election.cc:302] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b4c9cd7083b46b3b9d9df66473a4846, c68e5972701347c3a3f2c2516c7313b5; no voters:
I20260806 18:35:07.275578 26819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 18:35:07.275666 26819 raft_consensus.cc:482] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:35:07.275727 26819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:35:07.275813 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.275554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48576)
0806 18:35:07.275672 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.275726 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 18:35:07.275966 26029 leader_election.cc:334] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437): Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
I20260806 18:35:07.277006 26819 raft_consensus.cc:504] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } }
I20260806 18:35:07.277426 26819 leader_election.cc:288] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 3 election: Requested vote from peers 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437), 2b4c9cd7083b46b3b9d9df66473a4846 (127.18.125.188:38057)
I20260806 18:35:07.277760 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.277526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48588)
0806 18:35:07.277626 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.277685 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 18:35:07.277868 26248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c68e5972701347c3a3f2c2516c7313b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2b4c9cd7083b46b3b9d9df66473a4846"
I20260806 18:35:07.278031 26248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:35:07.278405 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.278213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48596)
0806 18:35:07.278304 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.278353 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 18:35:07.278522 26029 leader_election.cc:334] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437): Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
I20260806 18:35:07.279214 26248 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c68e5972701347c3a3f2c2516c7313b5 in term 3.
I20260806 18:35:07.279573 26031 leader_election.cc:302] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2b4c9cd7083b46b3b9d9df66473a4846, c68e5972701347c3a3f2c2516c7313b5; no voters: 630b9ddf4d484387a5769bb7c7f07d5c
I20260806 18:35:07.279811 26819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:35:07.280011 26819 raft_consensus.cc:686] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [term 3 LEADER]: Becoming Leader. State: Replica: c68e5972701347c3a3f2c2516c7313b5, State: Running, Role: LEADER
I20260806 18:35:07.280325 26819 consensus_queue.cc:237] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } }
I20260806 18:35:07.281544 26822 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c68e5972701347c3a3f2c2516c7313b5. Latest consensus state: current_term: 3 leader_uuid: "c68e5972701347c3a3f2c2516c7313b5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:07.281695 26822 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:07.282033 26824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:07.284303 26824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:07.284708 26824 catalog_manager.cc:1259] Loaded cluster ID: e60b7aa24ba2482a9ef3925ff72b8032
I20260806 18:35:07.284755 26824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:07.285349 26824 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:07.285746 26824 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: Loaded TSK: 0
I20260806 18:35:07.286510 26824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:35:07.363431 26046 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:35:07.366819 26046 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:35:07.370209 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.369930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48606)
0806 18:35:07.370071 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.370132 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 18:35:07.370600 26248 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [term 3 FOLLOWER]: Refusing update from remote peer c68e5972701347c3a3f2c2516c7313b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 18:35:07.371073 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.370782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.0.0.1:48610)
0806 18:35:07.370921 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.370990 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 18:35:07.371320 26029 consensus_peers.cc:487] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 -> Peer 630b9ddf4d484387a5769bb7c7f07d5c (127.18.125.190:42437): Couldn't send request to peer 630b9ddf4d484387a5769bb7c7f07d5c. Status: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 18:35:07.371335 26822 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260806 18:35:07.373278 26826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c68e5972701347c3a3f2c2516c7313b5. Latest consensus state: current_term: 3 leader_uuid: "c68e5972701347c3a3f2c2516c7313b5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:07.373430 26826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:35:07.374825 26822 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c68e5972701347c3a3f2c2516c7313b5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:07.374987 26822 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:07.375078 26826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c68e5972701347c3a3f2c2516c7313b5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "630b9ddf4d484387a5769bb7c7f07d5c" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42437 } } peers { permanent_uuid: "c68e5972701347c3a3f2c2516c7313b5" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 41833 } attrs { promote: false } } peers { permanent_uuid: "2b4c9cd7083b46b3b9d9df66473a4846" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38057 } attrs { promote: false } } }
I20260806 18:35:07.375206 26826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4c9cd7083b46b3b9d9df66473a4846 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:35:07.375595 26828 mvcc.cc:205] Tried to move back new op lower bound from 7315625194982715392 to 7315625194622861312. Current Snapshot: MvccSnapshot[applied={T|T < 7315625194982715392 or (T in {7315625194982715392})}]
I20260806 18:35:07.387778 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.382529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:36445 (local address 127.0.0.1:57892)
0806 18:35:07.383093 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.383111 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:07.383200 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:07.383935 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:07.383938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:07.383955 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:07.384093 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:07.384100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.385157 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.385161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:07.386430 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:07.386447 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.387307 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.387311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:07.387402 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:07.387429 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:35:07.387507 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 18:35:07.387585 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477}
I20260806 18:35:07.387794 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.383394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57892 (local address 127.18.125.131:36445)
0806 18:35:07.383632 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:07.383639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:07.383654 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:35:07.383712 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:07.383715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:07.383817 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:07.383912 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:07.384265 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.385024 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.386550 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.387169 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.387299 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:07.387358 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:07.387537 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 18:35:07.387622 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":94,"threads_started":1}
I20260806 18:35:07.388582 26659 tablet_service.cc:1467] Processing CreateTablet for tablet 47647accdcfe445184c2289960b0edfc (DEFAULT_TABLE table=test-table [id=1eac47a2174e465e8c6c4dd206cdd743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:07.388938 26659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47647accdcfe445184c2289960b0edfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:07.389153 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.383175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:41241 (local address 127.0.0.1:33898)
0806 18:35:07.383414 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.383425 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:07.383526 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:07.384098 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:07.384103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:07.384155 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:07.384322 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:07.384329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.386383 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.386386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:07.387649 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:07.387660 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.388778 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.388781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:07.388852 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:07.388883 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:35:07.388955 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:35:07.389005 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":98,"spinlock_wait_cycles":1920,"thread_start_us":93,"threads_started":1}
I20260806 18:35:07.389171 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.383558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33898 (local address 127.18.125.129:41241)
0806 18:35:07.383836 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:07.383839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:07.383851 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:35:07.383927 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:07.383930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:07.383979 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:07.384069 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:07.384428 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.386269 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.388072 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.388672 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.388806 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:07.388871 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:07.388972 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 18:35:07.389047 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":123,"threads_started":1}
I20260806 18:35:07.389304 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.382978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47270 (local address 127.18.125.130:33231)
0806 18:35:07.383809 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:07.383814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:07.383828 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:35:07.383885 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:07.383889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:07.383940 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:07.384027 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:07.384929 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.385682 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.387748 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.388466 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.388592 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:07.388650 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:07.389062 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 18:35:07.389146 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":72,"threads_started":1}
I20260806 18:35:07.389492 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.382816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:33231 (local address 127.0.0.1:47270)
0806 18:35:07.383127 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.383142 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:07.383239 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:07.384162 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:07.384165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:07.384188 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:07.384314 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:07.384324 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.386604 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.386608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:07.387611 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:07.387619 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.388715 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.388722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:07.388813 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:07.388837 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:35:07.388912 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:35:07.388996 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":8,"spinlock_wait_cycles":768,"thread_start_us":137,"threads_started":1}
I20260806 18:35:07.389750 26357 tablet_service.cc:1467] Processing CreateTablet for tablet 47647accdcfe445184c2289960b0edfc (DEFAULT_TABLE table=test-table [id=1eac47a2174e465e8c6c4dd206cdd743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:07.390023 26357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47647accdcfe445184c2289960b0edfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:07.390000 26504 tablet_service.cc:1467] Processing CreateTablet for tablet 47647accdcfe445184c2289960b0edfc (DEFAULT_TABLE table=test-table [id=1eac47a2174e465e8c6c4dd206cdd743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:07.390241 26504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47647accdcfe445184c2289960b0edfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:07.391381 26838 tablet_bootstrap.cc:492] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438: Bootstrap starting.
I20260806 18:35:07.392469 26838 tablet_bootstrap.cc:654] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:07.392877 26839 tablet_bootstrap.cc:492] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2: Bootstrap starting.
I20260806 18:35:07.393816 26839 tablet_bootstrap.cc:654] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:07.393949 26838 tablet_bootstrap.cc:492] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438: No bootstrap required, opened a new log
I20260806 18:35:07.394050 26838 ts_tablet_manager.cc:1397] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:07.394461 26838 raft_consensus.cc:348] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:07.394599 26838 raft_consensus.cc:374] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:07.394569 26840 tablet_bootstrap.cc:492] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9: Bootstrap starting.
I20260806 18:35:07.394637 26838 raft_consensus.cc:729] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe73b370c34842228aa10e9f0a150438, State: Initialized, Role: FOLLOWER
I20260806 18:35:07.394845 26838 consensus_queue.cc:260] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:07.395443 26838 ts_tablet_manager.cc:1428] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:35:07.395864 26840 tablet_bootstrap.cc:654] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:07.395970 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.395566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.131:42143)
0806 18:35:07.395817 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.395882 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
I20260806 18:35:07.396981 26839 tablet_bootstrap.cc:492] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2: No bootstrap required, opened a new log
I20260806 18:35:07.397136 26839 ts_tablet_manager.cc:1397] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 18:35:07.397467 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.397182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.131:51529)
0806 18:35:07.397339 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.397393 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 18:35:07.397676 26839 raft_consensus.cc:348] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:07.397884 26839 raft_consensus.cc:374] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:07.397910 26732 heartbeater.cc:503] Master 127.18.125.189:41833 was elected leader, sending a full tablet report...
I20260806 18:35:07.397974 26839 raft_consensus.cc:729] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f443fc0a32ec415297569cab7e77e2a2, State: Initialized, Role: FOLLOWER
I20260806 18:35:07.398136 26839 consensus_queue.cc:260] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:07.398746 26840 tablet_bootstrap.cc:492] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9: No bootstrap required, opened a new log
I20260806 18:35:07.398850 26840 ts_tablet_manager.cc:1397] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 18:35:07.399317 26840 raft_consensus.cc:348] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:07.399416 26840 raft_consensus.cc:374] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:07.399438 26840 raft_consensus.cc:729] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd88c02290fb40238ca877dda174f5a9, State: Initialized, Role: FOLLOWER
I20260806 18:35:07.399533 26840 consensus_queue.cc:260] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:07.399852 26840 ts_tablet_manager.cc:1428] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:35:07.400249 26839 ts_tablet_manager.cc:1428] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:35:07.400472 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.400160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.130:48567)
0806 18:35:07.400327 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.400386 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 18:35:07.401266 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.401007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.130:39493)
0806 18:35:07.401156 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.401206 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 18:35:07.402209 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.401967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.129:51209)
0806 18:35:07.402079 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.402138 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 18:35:07.403004 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.402749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.129:45687)
0806 18:35:07.402891 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.402941 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 18:35:07.403036 26580 heartbeater.cc:503] Master 127.18.125.189:41833 was elected leader, sending a full tablet report...
I20260806 18:35:07.403596 26427 heartbeater.cc:503] Master 127.18.125.189:41833 was elected leader, sending a full tablet report...
I20260806 18:35:07.476238 26843 raft_consensus.cc:482] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:35:07.476439 26843 raft_consensus.cc:504] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:07.476922 26843 leader_election.cc:288] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd88c02290fb40238ca877dda174f5a9 (127.18.125.130:33231), f443fc0a32ec415297569cab7e77e2a2 (127.18.125.129:41241)
I20260806 18:35:07.477535 26526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47647accdcfe445184c2289960b0edfc" candidate_uuid: "fe73b370c34842228aa10e9f0a150438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd88c02290fb40238ca877dda174f5a9" is_pre_election: true
I20260806 18:35:07.477596 26379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47647accdcfe445184c2289960b0edfc" candidate_uuid: "fe73b370c34842228aa10e9f0a150438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f443fc0a32ec415297569cab7e77e2a2" is_pre_election: true
I20260806 18:35:07.477710 26526 raft_consensus.cc:2454] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe73b370c34842228aa10e9f0a150438 in term 0.
I20260806 18:35:07.477800 26379 raft_consensus.cc:2454] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe73b370c34842228aa10e9f0a150438 in term 0.
I20260806 18:35:07.478113 26608 leader_election.cc:302] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd88c02290fb40238ca877dda174f5a9, fe73b370c34842228aa10e9f0a150438; no voters:
I20260806 18:35:07.478350 26843 raft_consensus.cc:2781] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:35:07.478410 26843 raft_consensus.cc:482] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:35:07.478437 26843 raft_consensus.cc:3037] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:07.479532 26843 raft_consensus.cc:504] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:07.479959 26843 leader_election.cc:288] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [CANDIDATE]: Term 1 election: Requested vote from peers dd88c02290fb40238ca877dda174f5a9 (127.18.125.130:33231), f443fc0a32ec415297569cab7e77e2a2 (127.18.125.129:41241)
I20260806 18:35:07.480499 26526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47647accdcfe445184c2289960b0edfc" candidate_uuid: "fe73b370c34842228aa10e9f0a150438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd88c02290fb40238ca877dda174f5a9"
I20260806 18:35:07.480499 26379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47647accdcfe445184c2289960b0edfc" candidate_uuid: "fe73b370c34842228aa10e9f0a150438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f443fc0a32ec415297569cab7e77e2a2"
I20260806 18:35:07.480644 26379 raft_consensus.cc:3037] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:07.480644 26526 raft_consensus.cc:3037] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:07.481837 26526 raft_consensus.cc:2454] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe73b370c34842228aa10e9f0a150438 in term 1.
I20260806 18:35:07.481837 26379 raft_consensus.cc:2454] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe73b370c34842228aa10e9f0a150438 in term 1.
I20260806 18:35:07.482152 26608 leader_election.cc:302] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd88c02290fb40238ca877dda174f5a9, fe73b370c34842228aa10e9f0a150438; no voters:
I20260806 18:35:07.482349 26843 raft_consensus.cc:2781] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:07.482583 26843 raft_consensus.cc:686] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 1 LEADER]: Becoming Leader. State: Replica: fe73b370c34842228aa10e9f0a150438, State: Running, Role: LEADER
I20260806 18:35:07.482748 26843 consensus_queue.cc:237] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } } peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } }
I20260806 18:35:07.483457 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.483159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.131:42035)
0806 18:35:07.483273 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.483337 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 18:35:07.485261 26046 catalog_manager.cc:5495] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe73b370c34842228aa10e9f0a150438 (127.18.125.131). New cstate: current_term: 1 leader_uuid: "fe73b370c34842228aa10e9f0a150438" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe73b370c34842228aa10e9f0a150438" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 } health_report { overall_health: UNKNOWN } } }
I20260806 18:35:07.516721 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.513307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57906 (local address 127.18.125.131:36445)
0806 18:35:07.513473 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:07.513477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:07.513489 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:35:07.513572 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:07.513574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:07.513618 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:07.513684 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:07.514000 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.514626 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.515643 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:07.515865 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:07.515946 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:07.516255 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:07.516330 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:07.516389 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:07.516484 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 18:35:07.516536 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 18:35:07.519105 26379 raft_consensus.cc:1261] T 47647accdcfe445184c2289960b0edfc P f443fc0a32ec415297569cab7e77e2a2 [term 1 FOLLOWER]: Refusing update from remote peer fe73b370c34842228aa10e9f0a150438: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:07.519143 26526 raft_consensus.cc:1261] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9 [term 1 FOLLOWER]: Refusing update from remote peer fe73b370c34842228aa10e9f0a150438: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:07.519608 26843 consensus_queue.cc:1035] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f443fc0a32ec415297569cab7e77e2a2" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 41241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:07.519819 26847 consensus_queue.cc:1035] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd88c02290fb40238ca877dda174f5a9" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:07.522149 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.521756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.130:49803)
0806 18:35:07.521970 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.522060 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260806 18:35:07.523255 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.522376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.129:47983)
0806 18:35:07.523099 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.523168 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":647}
I20260806 18:35:07.524432 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.523732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.131:54725)
0806 18:35:07.524296 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.524356 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488}
I20260806 18:35:07.526176 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.525787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.129:38425)
0806 18:35:07.526048 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.526102 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260806 18:35:07.526755 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.526505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.131:37607)
0806 18:35:07.526633 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.526690 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 18:35:07.527830 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.527517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42437 (local address 127.18.125.130:34595)
0806 18:35:07.527700 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.527756 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260806 18:35:07.529289 26852 mvcc.cc:205] Tried to move back new op lower bound from 7315625195593621504 to 7315625195453054976. Current Snapshot: MvccSnapshot[applied={T|T < 7315625195593621504 or (T in {7315625195593621504})}]
I20260806 18:35:07.531921 26046 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:07.532059 26046 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:07.536223 26046 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c68e5972701347c3a3f2c2516c7313b5: Sending DeleteTablet for 3 replicas of tablet 47647accdcfe445184c2289960b0edfc
I20260806 18:35:07.537117 26504 tablet_service.cc:1514] Processing DeleteTablet for tablet 47647accdcfe445184c2289960b0edfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:07 UTC) from {username='oryx', principal='oryx/127.18.125.189@KRBTEST.COM'} at 127.0.0.1:47270
I20260806 18:35:07.537145 26659 tablet_service.cc:1514] Processing DeleteTablet for tablet 47647accdcfe445184c2289960b0edfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:07 UTC) from {username='oryx', principal='oryx/127.18.125.189@KRBTEST.COM'} at 127.0.0.1:57892
I20260806 18:35:07.537202 26357 tablet_service.cc:1514] Processing DeleteTablet for tablet 47647accdcfe445184c2289960b0edfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:07 UTC) from {username='oryx', principal='oryx/127.18.125.189@KRBTEST.COM'} at 127.0.0.1:33898
I20260806 18:35:07.537648 26861 tablet_replica.cc:330] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9: stopping tablet replica
I20260806 18:35:07.537804 26861 raft_consensus.cc:2229] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:07.537973 26861 raft_consensus.cc:2258] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:07.538107 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 26194
I20260806 18:35:07.538267 26860 tablet_replica.cc:330] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438: stopping tablet replica
I20260806 18:35:07.538417 26860 raft_consensus.cc:2229] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:35:07.539175 26860 raft_consensus.cc:2258] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:07.539927 26861 ts_tablet_manager.cc:1905] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:07.540038 26860 ts_tablet_manager.cc:1905] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:07.543371 26860 ts_tablet_manager.cc:1918] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:07.543478 26860 log.cc:1196] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/47647accdcfe445184c2289960b0edfc
I20260806 18:35:07.543763 26860 ts_tablet_manager.cc:1939] T 47647accdcfe445184c2289960b0edfc P fe73b370c34842228aa10e9f0a150438: Deleting consensus metadata
I20260806 18:35:07.543939 26861 ts_tablet_manager.cc:1918] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:07.544060 26861 log.cc:1196] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/47647accdcfe445184c2289960b0edfc
I20260806 18:35:07.544480 26861 ts_tablet_manager.cc:1939] T 47647accdcfe445184c2289960b0edfc P dd88c02290fb40238ca877dda174f5a9: Deleting consensus metadata
I20260806 18:35:07.545468 26031 catalog_manager.cc:4841] TS dd88c02290fb40238ca877dda174f5a9 (127.18.125.130:33231): tablet 47647accdcfe445184c2289960b0edfc (table test-table [id=1eac47a2174e465e8c6c4dd206cdd743]) successfully deleted
I20260806 18:35:07.546763 26031 catalog_manager.cc:4841] TS fe73b370c34842228aa10e9f0a150438 (127.18.125.131:36445): tablet 47647accdcfe445184c2289960b0edfc (table test-table [id=1eac47a2174e465e8c6c4dd206cdd743]) successfully deleted
I20260806 18:35:07.553473 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 26433
I20260806 18:35:07.553663 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:07.553334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:41241 (local address 127.0.0.1:33910)
0806 18:35:07.553485 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:07.553543 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:41241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 18:35:07.555226 26031 catalog_manager.cc:4584] TS f443fc0a32ec415297569cab7e77e2a2 (127.18.125.129:41241): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47647accdcfe445184c2289960b0edfc: Network error: Client connection negotiation failed: client connection to 127.18.125.129:41241: connect: Connection refused (error 111)
I20260806 18:35:07.561789 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 26585
I20260806 18:35:07.570163 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 25902
I20260806 18:35:07.579015 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 26008
2026-08-06T18:35:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7429 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:35:07 dist-test-slave-8w9v krb5kdc[26865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:35:07 dist-test-slave-8w9v krb5kdc[26865](info): set up 2 sockets
Aug 06 18:35:07 dist-test-slave-8w9v krb5kdc[26865](info): commencing operation
krb5kdc: starting...
W20260806 18:35:08.718930 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:35:08 dist-test-slave-8w9v krb5kdc[26865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:35:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.190.keytab.
Entry for principal oryx/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.190.keytab.
I20260806 18:35:08.856678 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:41669
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.190.keytab
--principal=oryx/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:41669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:08.976097 26878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:08.976500 26878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:08.976547 26878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:08.980666 26878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:08.980741 26878 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:35:08.980762 26878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:08.980784 26878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:08.980804 26878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:08.980822 26878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:08.986267 26878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:41669
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.190.keytab
--principal=oryx/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:41669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:08.987320 26878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:08.988350 26878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:08.997658 26885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:08.997671 26878 server_base.cc:1034] running on GCE node
W20260806 18:35:08.997627 26884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:08.997627 26887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:08.998523 26878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:08.999719 26878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:09.000885 26878 hybrid_clock.cc:648] HybridClock initialized: now 1786041309000867 us; error 30 us; skew 500 ppm
Aug 06 18:35:08 dist-test-slave-8w9v krb5kdc[26865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:09.003966 26878 init.cc:377] Logged in from keytab as oryx/127.18.125.190@KRBTEST.COM (short username oryx)
I20260806 18:35:09.005090 26878 webserver.cc:460] Webserver started at http://127.18.125.190:39413/ using document root <none> and password file <none>
I20260806 18:35:09.005764 26878 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:09.005865 26878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:09.006124 26878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:09.008445 26878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "9e06741026684d59a5b3683f3b62e760"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "718e6b90a38e4802c882ec73156806e7"
server_key_iv: "1dc2d0112fb71f5832d88bba6afdb94d"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.009085 26878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "9e06741026684d59a5b3683f3b62e760"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "718e6b90a38e4802c882ec73156806e7"
server_key_iv: "1dc2d0112fb71f5832d88bba6afdb94d"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.013634 26878 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:35:09.016492 26894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:09.017874 26878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:09.018042 26878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "9e06741026684d59a5b3683f3b62e760"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "718e6b90a38e4802c882ec73156806e7"
server_key_iv: "1dc2d0112fb71f5832d88bba6afdb94d"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.018177 26878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:09.046897 26878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:09.050194 26878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:09.050421 26878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:09.059248 26878 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:41669
I20260806 18:35:09.059319 26958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:41669 every 8 connection(s)
I20260806 18:35:09.060344 26878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:35:09.063155 26959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:09.064440 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 26878
I20260806 18:35:09.064599 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:35:09.064924 18934 external_mini_cluster.cc:1428] Setting key 5ba441ba89a46228e2a8c6593f422ccd
I20260806 18:35:09.069657 26959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760: Bootstrap starting.
I20260806 18:35:09.072050 26959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:09.072855 26959 log.cc:824] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:35:09 dist-test-slave-8w9v krb5kdc[26865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
I20260806 18:35:09.075161 26959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760: No bootstrap required, opened a new log
I20260806 18:35:09.079099 26959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e06741026684d59a5b3683f3b62e760" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41669 } }
I20260806 18:35:09.079284 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.066356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.18.125.190:41669)
0806 18:35:09.066851 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.066866 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.066903 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:35:09.067553 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.067570 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.067906 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.068291 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:09.069122 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.070147 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.070985 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.071461 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.071639 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.074430 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:09.074435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:09.074449 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:09.074477 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:09.076372 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.076850 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.076855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.076859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.076976 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.077271 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.077275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.077278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.077678 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:09.077912 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.078216 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.078520 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":150,"threads_started":1}
I20260806 18:35:09.079334 26959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:09.079397 26959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e06741026684d59a5b3683f3b62e760, State: Initialized, Role: FOLLOWER
I20260806 18:35:09.079821 26959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e06741026684d59a5b3683f3b62e760" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41669 } }
I20260806 18:35:09.079995 26959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:35:09.080080 26959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:35:09.080231 26959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:09.081552 26959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e06741026684d59a5b3683f3b62e760" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41669 } }
I20260806 18:35:09.081948 26959 leader_election.cc:302] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e06741026684d59a5b3683f3b62e760; no voters:
I20260806 18:35:09.082365 26959 leader_election.cc:288] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:35:09.082487 26964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:09.082717 26964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [term 1 LEADER]: Becoming Leader. State: Replica: 9e06741026684d59a5b3683f3b62e760, State: Running, Role: LEADER
I20260806 18:35:09.083078 26964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e06741026684d59a5b3683f3b62e760" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41669 } }
I20260806 18:35:09.083665 26959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:09.084952 26965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e06741026684d59a5b3683f3b62e760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e06741026684d59a5b3683f3b62e760" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41669 } } }
I20260806 18:35:09.085088 26965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:09.085443 26966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e06741026684d59a5b3683f3b62e760. Latest consensus state: current_term: 1 leader_uuid: "9e06741026684d59a5b3683f3b62e760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e06741026684d59a5b3683f3b62e760" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 41669 } } }
I20260806 18:35:09.085551 26966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760 [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:09.085522 26973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:09.088992 26973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:09.095115 26973 catalog_manager.cc:1347] Generated new cluster ID: 24637f27677e4b7d89328cc69fb38d5e
I20260806 18:35:09.095216 26973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:09.114305 26973 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:35:09.115481 26973 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:09.122082 26973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760: Generated new TSK 0
I20260806 18:35:09.123032 26973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab.
Entry for principal oryx/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab.
I20260806 18:35:09.160903 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:41669
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab
--principal=oryx/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:09.285895 26985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:09.286222 26985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:09.286310 26985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:09.291723 26985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:09.291855 26985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:09.291945 26985 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:35:09.292039 26985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:35:09.298519 26985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab
--principal=oryx/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.125.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:09.299782 26985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:09.301031 26985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:09.312031 26992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:09.312031 26995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:09.312031 26991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:09.312381 26985 server_base.cc:1034] running on GCE node
I20260806 18:35:09.312939 26985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:09.313660 26985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:09.314870 26985 hybrid_clock.cc:648] HybridClock initialized: now 1786041309314839 us; error 44 us; skew 500 ppm
Aug 06 18:35:09 dist-test-slave-8w9v krb5kdc[26865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:09.317868 26985 init.cc:377] Logged in from keytab as oryx/127.18.125.129@KRBTEST.COM (short username oryx)
I20260806 18:35:09.318801 26985 webserver.cc:460] Webserver started at http://127.18.125.129:39221/ using document root <none> and password file <none>
I20260806 18:35:09.319406 26985 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:09.319497 26985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:09.319732 26985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:09.322038 26985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "e18e0819d2cc4a548af63bfb501bbab8"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "f6d1e1be47aa86bf453d09ba34ad0ea7"
server_key_iv: "40301897043e5e64ee53640fec9da808"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.322618 26985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "e18e0819d2cc4a548af63bfb501bbab8"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "f6d1e1be47aa86bf453d09ba34ad0ea7"
server_key_iv: "40301897043e5e64ee53640fec9da808"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.326601 26985 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:35:09.329470 27003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:09.330742 26985 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:35:09.330915 26985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "e18e0819d2cc4a548af63bfb501bbab8"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "f6d1e1be47aa86bf453d09ba34ad0ea7"
server_key_iv: "40301897043e5e64ee53640fec9da808"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.331038 26985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:09.347122 26985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:09.350464 26985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:09.350679 26985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:09.352216 26985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:09.352283 26985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:09.352346 26985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:09.352394 26985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:35:09 dist-test-slave-8w9v krb5kdc[26865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
I20260806 18:35:09.363832 26985 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:44399
I20260806 18:35:09.363917 27135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:44399 every 8 connection(s)
I20260806 18:35:09.364989 26985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:09.367043 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.353185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:35909 (local address 127.18.125.190:41669)
0806 18:35:09.353385 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.353391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.354197 (+ 806us) server_negotiation.cc:407] Connection header received
0806 18:35:09.355277 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.355281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.355363 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.355425 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:09.357133 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.357858 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.359068 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.359548 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.359690 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.363040 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:09.363044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:09.363046 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:09.363075 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:09.364742 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.365383 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.365389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.365391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.365494 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.365935 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.365939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.365941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.366191 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:09.366281 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.366809 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.366920 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:35:09.367933 27013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.353553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:41669 (local address 127.18.125.129:35909)
0806 18:35:09.354017 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.354070 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:09.355021 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:09.355887 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:09.355894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:09.356387 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:09.356961 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.356995 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.357995 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.357998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.358899 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.358909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.359754 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.359759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.359984 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.360835 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:09.360852 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:09.362818 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:09.364869 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.364875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.364891 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.365242 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.365617 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.365620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.365622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.365781 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.366327 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:09.366333 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:09.366568 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 18:35:09.366739 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 18:35:09.367012 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":128,"threads_started":1}
I20260806 18:35:09.369396 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 26985
I20260806 18:35:09.369505 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:35:09.369725 27136 heartbeater.cc:347] Connected to a master server at 127.18.125.190:41669
I20260806 18:35:09.369870 18934 external_mini_cluster.cc:1428] Setting key dcfbcb946d80ac956f1723901e87248d
I20260806 18:35:09.370092 27136 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:09.371059 27136 heartbeater.cc:511] Master 127.18.125.190:41669 requested a full tablet report, sending...
I20260806 18:35:09.372812 26910 ts_manager.cc:194] Registered new tserver with Master: e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129:44399)
I20260806 18:35:09.374591 26910 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:35909
WARNING: no policy specified for oryx/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:35:09.382922 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.372664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:57727 (local address 127.18.125.190:41669)
0806 18:35:09.372804 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.372811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.373473 (+ 662us) server_negotiation.cc:407] Connection header received
0806 18:35:09.373585 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.373591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.373653 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.373745 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:09.374379 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.375211 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.376942 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.377273 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.377392 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.379042 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:09.379047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:09.379050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:09.379096 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:09.380875 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.381467 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.381469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.381471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.381561 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.381905 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.381907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.381909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.382121 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:09.382205 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.382607 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.382727 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":45}
I20260806 18:35:09.382987 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.372930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:41669 (local address 127.18.125.129:57727)
0806 18:35:09.373354 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.373371 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:09.373495 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:09.373798 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:09.373801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:09.374093 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:09.374233 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.374241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.375420 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.375424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.376777 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.376796 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.377438 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.377445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.377597 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.378196 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:09.378217 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:09.378752 (+ 535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:09.381012 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.381018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.381023 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.381344 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.381670 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.381673 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.381675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.381783 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.382239 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:09.382244 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:09.382385 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 18:35:09.382517 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 18:35:09.382689 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":127,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab.
Entry for principal oryx/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab.
I20260806 18:35:09.403600 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:41669
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab
--principal=oryx/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:09.523931 27147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:09.524279 27147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:09.524333 27147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:09.528330 27147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:09.528391 27147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:09.528434 27147 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:35:09.528466 27147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:35:09.534024 27147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab
--principal=oryx/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.125.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:09.535171 27147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:09.536355 27147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:09.546291 27153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:09.546291 27152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:09.546685 27155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:09.546624 27147 server_base.cc:1034] running on GCE node
I20260806 18:35:09.547204 27147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:09.547859 27147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:09.549038 27147 hybrid_clock.cc:648] HybridClock initialized: now 1786041309549014 us; error 38 us; skew 500 ppm
Aug 06 18:35:09 dist-test-slave-8w9v krb5kdc[26865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:09.552006 27147 init.cc:377] Logged in from keytab as oryx/127.18.125.130@KRBTEST.COM (short username oryx)
I20260806 18:35:09.553012 27147 webserver.cc:460] Webserver started at http://127.18.125.130:37217/ using document root <none> and password file <none>
I20260806 18:35:09.553637 27147 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:09.553754 27147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:09.554024 27147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:09.556435 27147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "f1ff8761a16d4560b2688ff332e3b6b6"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "edefcec41ea7617de36519d6f6f854e1"
server_key_iv: "6c682160dcbf01f24c12663df3be39a1"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.557118 27147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "f1ff8761a16d4560b2688ff332e3b6b6"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "edefcec41ea7617de36519d6f6f854e1"
server_key_iv: "6c682160dcbf01f24c12663df3be39a1"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.561651 27147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:35:09.564553 27162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:09.565949 27147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:35:09.566115 27147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "f1ff8761a16d4560b2688ff332e3b6b6"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "edefcec41ea7617de36519d6f6f854e1"
server_key_iv: "6c682160dcbf01f24c12663df3be39a1"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.566268 27147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:09.583572 27147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:09.587816 27147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:09.588069 27147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:09.589816 27147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:09.589905 27147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:09.589989 27147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:09.590050 27147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:35:09 dist-test-slave-8w9v krb5kdc[26865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
I20260806 18:35:09.602722 27147 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:43141
I20260806 18:35:09.602772 27290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:43141 every 8 connection(s)
I20260806 18:35:09.604017 27147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:09.605576 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.590729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:38023 (local address 127.18.125.190:41669)
0806 18:35:09.590863 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.590867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.592057 (+ 1190us) server_negotiation.cc:407] Connection header received
0806 18:35:09.593201 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.593205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.593278 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.593390 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:09.595160 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.596048 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.597274 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.597518 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.597657 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.601163 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:09.601167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:09.601170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:09.601205 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:09.603073 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.603620 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.603626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.603631 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.603755 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.604183 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.604187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.604189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.604489 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:09.604635 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.605313 (+ 678us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.605441 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:35:09.606455 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.591345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:41669 (local address 127.18.125.130:38023)
0806 18:35:09.591879 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.591924 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:09.592947 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:09.593718 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:09.593747 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:09.594297 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:09.595006 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.595031 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.596193 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.596197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.597134 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.597144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.597722 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.597727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.597940 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.598678 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:09.598704 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:09.600964 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:09.603198 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.603203 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.603214 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.603482 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.603872 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.603874 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.603875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.604020 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.604638 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:09.604645 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:09.604952 (+ 307us) client_negotiation.cc:769] Sending connection context
0806 18:35:09.605178 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 18:35:09.605522 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":143,"threads_started":1}
I20260806 18:35:09.607849 27291 heartbeater.cc:347] Connected to a master server at 127.18.125.190:41669
I20260806 18:35:09.607995 27291 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:09.608614 27291 heartbeater.cc:511] Master 127.18.125.190:41669 requested a full tablet report, sending...
I20260806 18:35:09.609908 26911 ts_manager.cc:194] Registered new tserver with Master: f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141)
I20260806 18:35:09.610673 26911 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:38023
I20260806 18:35:09.611552 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 27147
I20260806 18:35:09.611692 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:35:09.611958 18934 external_mini_cluster.cc:1428] Setting key c7c5e4ee348d4b57c94f33fcdcd27ecb
I20260806 18:35:09.618408 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.610341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:41669 (local address 127.18.125.130:49247)
0806 18:35:09.610652 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.610665 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:09.610748 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:09.611098 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:09.611101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:09.611349 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:09.611508 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.611516 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.612572 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.612576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.613223 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.613233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.613811 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.613813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.613884 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.614441 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:09.614452 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:09.614764 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:09.616697 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.616700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.616702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.616890 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.617188 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.617190 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.617192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.617264 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.617824 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:09.617826 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:09.617917 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 18:35:09.618032 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 18:35:09.618234 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":136,"threads_started":1}
I20260806 18:35:09.618500 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.610391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:49247 (local address 127.18.125.190:41669)
0806 18:35:09.610518 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.610522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.610736 (+ 214us) server_negotiation.cc:407] Connection header received
0806 18:35:09.610925 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.610928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.610983 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.611064 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:09.611649 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.612415 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.613373 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.613689 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.613827 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.614907 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:09.614913 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:09.614919 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:09.614952 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:09.616590 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.616987 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.616994 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.616998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.617091 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.617375 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.617381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.617386 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.617691 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:09.617806 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.618125 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.618257 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for oryx/127.18.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab.
Entry for principal oryx/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab.
I20260806 18:35:09.644755 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:41669
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab
--principal=oryx/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:09.767419 27301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:09.767807 27301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:09.767896 27301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:09.772764 27301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:09.772874 27301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:09.772955 27301 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:35:09.773049 27301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:35:09.780376 27301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab
--principal=oryx/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.125.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:09.781697 27301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:09.782825 27301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:09.794327 27306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:09.794327 27307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:09.794627 27301 server_base.cc:1034] running on GCE node
W20260806 18:35:09.794332 27309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:09.795219 27301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:09.795960 27301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:09.797158 27301 hybrid_clock.cc:648] HybridClock initialized: now 1786041309797126 us; error 42 us; skew 500 ppm
Aug 06 18:35:09 dist-test-slave-8w9v krb5kdc[26865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:09.800084 27301 init.cc:377] Logged in from keytab as oryx/127.18.125.131@KRBTEST.COM (short username oryx)
I20260806 18:35:09.801138 27301 webserver.cc:460] Webserver started at http://127.18.125.131:33641/ using document root <none> and password file <none>
I20260806 18:35:09.801757 27301 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:09.801859 27301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:09.802098 27301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:09.804497 27301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "9c528a6718464a6e80337257e704ce4e"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "b44ab74b993f76d381fc00565fad7590"
server_key_iv: "5d6402d3a7916113c96165a8536d5724"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.805253 27301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "9c528a6718464a6e80337257e704ce4e"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "b44ab74b993f76d381fc00565fad7590"
server_key_iv: "5d6402d3a7916113c96165a8536d5724"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.809671 27301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:35:09.813102 27316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:09.814668 27301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:35:09.814863 27301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "9c528a6718464a6e80337257e704ce4e"
format_stamp: "Formatted at 2026-08-06 18:35:09 on dist-test-slave-8w9v"
server_key: "b44ab74b993f76d381fc00565fad7590"
server_key_iv: "5d6402d3a7916113c96165a8536d5724"
server_key_version: "encryptionkey@0"
I20260806 18:35:09.815013 27301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:09.826188 27301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:09.829406 27301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:09.829645 27301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:09.831269 27301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:09.831362 27301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:09.831465 27301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:09.831516 27301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:35:09 dist-test-slave-8w9v krb5kdc[26865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
I20260806 18:35:09.844728 27301 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:44057
I20260806 18:35:09.844798 27446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:44057 every 8 connection(s)
I20260806 18:35:09.846077 27301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:09.847298 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.832096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:51505 (local address 127.18.125.190:41669)
0806 18:35:09.832266 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.832272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.833268 (+ 996us) server_negotiation.cc:407] Connection header received
0806 18:35:09.834641 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.834646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.834713 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.834820 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:09.836551 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.837215 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.839110 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.839416 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.839541 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.842788 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:09.842792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:09.842795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:09.842828 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:09.844551 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.845412 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.845416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.845418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.845545 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.845970 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.845974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.845976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.846237 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:09.846357 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.846995 (+ 638us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.847137 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:35:09.849161 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.832529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:41669 (local address 127.18.125.131:51505)
0806 18:35:09.833068 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.833127 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:09.834351 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:09.835022 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:09.835032 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:09.835587 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:09.836372 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.836396 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.837357 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.837365 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.838989 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.839005 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.839598 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.839603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.839790 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.840495 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:09.840512 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:09.842536 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:09.844693 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.844701 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.844729 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.845091 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.845641 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.845646 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.845649 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.845863 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.846358 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:09.846366 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:09.846692 (+ 326us) client_negotiation.cc:769] Sending connection context
0806 18:35:09.847976 (+ 1284us) client_negotiation.cc:240] Negotiation successful
0806 18:35:09.848333 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":125,"threads_started":1}
I20260806 18:35:09.852201 27447 heartbeater.cc:347] Connected to a master server at 127.18.125.190:41669
I20260806 18:35:09.852356 27447 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:09.852415 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 27301
I20260806 18:35:09.852552 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:35:09.852883 27447 heartbeater.cc:511] Master 127.18.125.190:41669 requested a full tablet report, sending...
I20260806 18:35:09.852977 18934 external_mini_cluster.cc:1428] Setting key 9e609d61b3155cf9abd62a7c75875fba
I20260806 18:35:09.853946 26911 ts_manager.cc:194] Registered new tserver with Master: 9c528a6718464a6e80337257e704ce4e (127.18.125.131:44057)
I20260806 18:35:09.854660 26911 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:51505
I20260806 18:35:09.855472 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:35:09.861716 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.853104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:36933 (local address 127.18.125.190:41669)
0806 18:35:09.853238 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.853242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.853953 (+ 711us) server_negotiation.cc:407] Connection header received
0806 18:35:09.854107 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.854110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.854163 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.854214 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:09.854906 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.855578 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.856498 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.856740 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.856851 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.858190 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:09.858193 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:09.858196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:09.858218 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:09.859755 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.860366 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.860369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.860370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.860448 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.860789 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.860791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.860793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.860986 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:09.861066 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.861397 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.861498 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":54}
I20260806 18:35:09.861791 27454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.853126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:41669 (local address 127.18.125.131:36933)
0806 18:35:09.853883 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.853900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:09.853995 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:09.854416 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:09.854419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:09.854643 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:09.854770 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.854778 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.855715 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.855717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.856381 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.856390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.856868 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.856871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.856968 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.857523 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:09.857535 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:09.858002 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:09.859877 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.859883 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.859888 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.860256 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.860550 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:09.860556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:09.860561 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:09.860672 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:09.861089 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:09.861094 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:09.861219 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 18:35:09.861343 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 18:35:09.861514 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":96,"threads_started":1}
Aug 06 18:35:09 dist-test-slave-8w9v krb5kdc[26865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041309, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:35:09 dist-test-slave-8w9v krb5kdc[26865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:09 dist-test-slave-8w9v krb5kdc[26865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:35:09.882375 26962 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:09.875727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36246 (local address 127.18.125.190:41669)
0806 18:35:09.875875 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.875878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.875894 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:35:09.875942 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.875944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.876007 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.876088 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:09.876858 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.877404 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.878304 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.878531 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.878625 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.882213 (+ 3588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36246: BlockingRecv error: recv got EOF from 127.0.0.1:36246 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 18:35:09 dist-test-slave-8w9v krb5kdc[26865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041309, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
I20260806 18:35:09.892709 26962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.884040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.18.125.190:41669)
0806 18:35:09.884208 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.884212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.884333 (+ 121us) server_negotiation.cc:407] Connection header received
0806 18:35:09.884503 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.884506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.884558 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.884644 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:09.885295 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.885816 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.886692 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.886913 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.886990 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.889361 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:09.889365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:09.889367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:09.889390 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:09.890870 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.891248 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.891251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.891252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.891325 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:09.891562 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:09.891564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:09.891565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:09.891747 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:09.891853 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.892355 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.892480 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:35:09.896698 26911 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:35:09.899822 26911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:35:09.916109 27471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.909199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44399 (local address 127.0.0.1:50650)
0806 18:35:09.910178 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.910193 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:09.910286 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:09.911010 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:09.911016 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:09.911055 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:09.911406 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.911413 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.912875 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.912878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.914441 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.914453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.915558 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.915561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.915661 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.915685 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:35:09.915740 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 18:35:09.915826 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":910,"mutex_wait_us":191,"spinlock_wait_cycles":9472,"thread_start_us":93,"threads_started":1}
I20260806 18:35:09.916138 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.909081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:43141 (local address 127.0.0.1:45150)
0806 18:35:09.909992 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.910049 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:09.910190 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:09.911010 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:09.911016 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:09.911054 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:09.911405 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.911413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.912792 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.912799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.914441 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.914453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.915357 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.915362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.915497 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.915540 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 18:35:09.915674 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 18:35:09.915833 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":260,"spinlock_wait_cycles":16128,"thread_start_us":66,"threads_started":1}
I20260806 18:35:09.916519 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.910258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.18.125.131:44057)
0806 18:35:09.910579 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.910590 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.911380 (+ 790us) server_negotiation.cc:407] Connection header received
0806 18:35:09.911549 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.911555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.911751 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.911921 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:09.912159 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.913542 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.914621 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.915405 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.915563 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.915725 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.916217 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.916335 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":136,"threads_started":1}
I20260806 18:35:09.916688 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.909817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:44057 (local address 127.0.0.1:48502)
0806 18:35:09.911254 (+ 1437us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:09.911267 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:09.911400 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:09.911914 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:09.911916 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:09.911936 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:09.912035 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.912040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.913676 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.913683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.914477 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:09.914486 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.915787 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.915790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:09.915885 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.915905 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:35:09.915990 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:35:09.916057 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1376,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260806 18:35:09.916981 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.909421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45150 (local address 127.18.125.130:43141)
0806 18:35:09.910495 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.910515 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.910536 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:35:09.910616 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.910621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.910832 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.911000 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:09.911556 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.912673 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.914597 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.915211 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.915397 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.915563 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.916657 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.916770 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":962,"thread_start_us":136,"threads_started":1}
I20260806 18:35:09.917745 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:09.909784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50650 (local address 127.18.125.129:44399)
0806 18:35:09.910495 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:09.910504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:09.910533 (+ 29us) server_negotiation.cc:407] Connection header received
0806 18:35:09.910615 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:09.910621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:09.910832 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:09.911000 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:09.911559 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.912690 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.914597 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:09.915405 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:09.917286 (+ 1881us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:09.917432 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:09.917484 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 18:35:09.917588 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":104,"threads_started":1}
I20260806 18:35:09.919981 27369 tablet_service.cc:1467] Processing CreateTablet for tablet 2579bf64fa494a5cb342906ff318daa2 (DEFAULT_TABLE table=test-table [id=b79ae5468f5f418a873576f6a42dd614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:09.919981 27216 tablet_service.cc:1467] Processing CreateTablet for tablet 2579bf64fa494a5cb342906ff318daa2 (DEFAULT_TABLE table=test-table [id=b79ae5468f5f418a873576f6a42dd614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:09.920256 27059 tablet_service.cc:1467] Processing CreateTablet for tablet 2579bf64fa494a5cb342906ff318daa2 (DEFAULT_TABLE table=test-table [id=b79ae5468f5f418a873576f6a42dd614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:09.921229 27369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2579bf64fa494a5cb342906ff318daa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:09.921229 27216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2579bf64fa494a5cb342906ff318daa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:09.921352 27059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2579bf64fa494a5cb342906ff318daa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:09.928195 27476 tablet_bootstrap.cc:492] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: Bootstrap starting.
I20260806 18:35:09.929750 27478 tablet_bootstrap.cc:492] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: Bootstrap starting.
I20260806 18:35:09.930176 27477 tablet_bootstrap.cc:492] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: Bootstrap starting.
I20260806 18:35:09.931591 27476 tablet_bootstrap.cc:654] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:09.931880 27478 tablet_bootstrap.cc:654] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:09.932112 27477 tablet_bootstrap.cc:654] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:09.932570 27476 log.cc:824] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:09.932747 27478 log.cc:824] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:09.932884 27477 log.cc:824] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:09.934886 27476 tablet_bootstrap.cc:492] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: No bootstrap required, opened a new log
I20260806 18:35:09.934888 27478 tablet_bootstrap.cc:492] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: No bootstrap required, opened a new log
I20260806 18:35:09.935094 27476 ts_tablet_manager.cc:1397] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 18:35:09.935094 27478 ts_tablet_manager.cc:1397] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:35:09.935632 27477 tablet_bootstrap.cc:492] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: No bootstrap required, opened a new log
I20260806 18:35:09.935981 27477 ts_tablet_manager.cc:1397] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 18:35:09.939560 27478 raft_consensus.cc:348] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:09.939767 27476 raft_consensus.cc:348] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:09.939987 27478 raft_consensus.cc:374] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:09.940030 27476 raft_consensus.cc:374] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:09.939832 27477 raft_consensus.cc:348] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:09.940073 27478 raft_consensus.cc:729] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ff8761a16d4560b2688ff332e3b6b6, State: Initialized, Role: FOLLOWER
I20260806 18:35:09.940104 27476 raft_consensus.cc:729] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18e0819d2cc4a548af63bfb501bbab8, State: Initialized, Role: FOLLOWER
I20260806 18:35:09.940212 27477 raft_consensus.cc:374] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:09.940294 27477 raft_consensus.cc:729] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c528a6718464a6e80337257e704ce4e, State: Initialized, Role: FOLLOWER
I20260806 18:35:09.940650 27478 consensus_queue.cc:260] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:09.940650 27476 consensus_queue.cc:260] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:09.940769 27477 consensus_queue.cc:260] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:09.941428 27136 heartbeater.cc:503] Master 127.18.125.190:41669 was elected leader, sending a full tablet report...
I20260806 18:35:09.941835 27476 ts_tablet_manager.cc:1428] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 18:35:09.941835 27478 ts_tablet_manager.cc:1428] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 18:35:09.941918 27477 ts_tablet_manager.cc:1428] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:35:09.942233 27291 heartbeater.cc:503] Master 127.18.125.190:41669 was elected leader, sending a full tablet report...
I20260806 18:35:09.942703 27447 heartbeater.cc:503] Master 127.18.125.190:41669 was elected leader, sending a full tablet report...
W20260806 18:35:10.097865 27448 tablet.cc:2369] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:35:10.106088 27292 tablet.cc:2369] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:35:10.116555 27137 tablet.cc:2369] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:35:10.273908 27484 raft_consensus.cc:482] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:35:10.274119 27484 raft_consensus.cc:504] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.275305 27484 leader_election.cc:288] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141), e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129:44399)
I20260806 18:35:10.279975 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.275728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:50695 (local address 127.18.125.129:44399)
0806 18:35:10.275850 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.275855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.275899 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:35:10.276099 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.276103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.276201 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.276295 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:10.276653 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.277450 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.278686 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.279312 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.279411 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.279459 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.279739 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.279814 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:35:10.279989 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.275594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44399 (local address 127.18.125.131:50695)
0806 18:35:10.275817 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:10.275837 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:10.275944 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:10.276352 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:10.276355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:10.276377 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:10.276526 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.276534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.277570 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.277573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.278552 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.278562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.279454 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.279458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.279595 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.279624 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:35:10.279710 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:35:10.279793 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260806 18:35:10.280479 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.275680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:49969 (local address 127.18.125.130:43141)
0806 18:35:10.275841 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.275847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.276230 (+ 383us) server_negotiation.cc:407] Connection header received
0806 18:35:10.276352 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.276356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.276405 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.276496 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:10.277281 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.277982 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.279206 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.279775 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.279900 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.279947 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.280229 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.280352 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:35:10.280870 27485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.275598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:43141 (local address 127.18.125.131:49969)
0806 18:35:10.276105 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:10.276143 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:10.276256 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:10.276520 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:10.276522 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:10.276538 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:10.277139 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.277150 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.278103 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.278105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.279067 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.279078 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.279907 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.279911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.280030 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.280061 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:35:10.280666 (+ 605us) client_negotiation.cc:240] Negotiation successful
0806 18:35:10.280742 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":68,"thread_start_us":145,"threads_started":1}
I20260806 18:35:10.281014 27081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2579bf64fa494a5cb342906ff318daa2" candidate_uuid: "9c528a6718464a6e80337257e704ce4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18e0819d2cc4a548af63bfb501bbab8" is_pre_election: true
I20260806 18:35:10.281427 27081 raft_consensus.cc:2454] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c528a6718464a6e80337257e704ce4e in term 0.
I20260806 18:35:10.281548 27238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2579bf64fa494a5cb342906ff318daa2" candidate_uuid: "9c528a6718464a6e80337257e704ce4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" is_pre_election: true
I20260806 18:35:10.281834 27238 raft_consensus.cc:2454] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c528a6718464a6e80337257e704ce4e in term 0.
I20260806 18:35:10.282038 27319 leader_election.cc:302] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c528a6718464a6e80337257e704ce4e, e18e0819d2cc4a548af63bfb501bbab8; no voters:
I20260806 18:35:10.282430 27484 raft_consensus.cc:2781] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:35:10.282563 27484 raft_consensus.cc:482] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:35:10.282625 27484 raft_consensus.cc:3037] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.283999 27484 raft_consensus.cc:504] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.284471 27484 leader_election.cc:288] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [CANDIDATE]: Term 1 election: Requested vote from peers f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141), e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129:44399)
I20260806 18:35:10.285006 27238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2579bf64fa494a5cb342906ff318daa2" candidate_uuid: "9c528a6718464a6e80337257e704ce4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ff8761a16d4560b2688ff332e3b6b6"
I20260806 18:35:10.285043 27081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2579bf64fa494a5cb342906ff318daa2" candidate_uuid: "9c528a6718464a6e80337257e704ce4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18e0819d2cc4a548af63bfb501bbab8"
I20260806 18:35:10.285133 27238 raft_consensus.cc:3037] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.285207 27081 raft_consensus.cc:3037] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.286381 27238 raft_consensus.cc:2454] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c528a6718464a6e80337257e704ce4e in term 1.
I20260806 18:35:10.286425 27081 raft_consensus.cc:2454] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c528a6718464a6e80337257e704ce4e in term 1.
I20260806 18:35:10.286818 27319 leader_election.cc:302] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c528a6718464a6e80337257e704ce4e, e18e0819d2cc4a548af63bfb501bbab8; no voters:
I20260806 18:35:10.287011 27484 raft_consensus.cc:2781] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:10.287281 27484 raft_consensus.cc:686] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 1 LEADER]: Becoming Leader. State: Replica: 9c528a6718464a6e80337257e704ce4e, State: Running, Role: LEADER
I20260806 18:35:10.287672 27484 consensus_queue.cc:237] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.291132 26909 catalog_manager.cc:5495] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c528a6718464a6e80337257e704ce4e (127.18.125.131). New cstate: current_term: 1 leader_uuid: "9c528a6718464a6e80337257e704ce4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } health_report { overall_health: HEALTHY } } }
I20260806 18:35:10.371295 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.366803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.18.125.131:44057)
0806 18:35:10.366954 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.366960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.366979 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:35:10.367133 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.367137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.367209 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.367338 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:10.367637 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.368555 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.369403 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.369810 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.369925 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.370308 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:10.370716 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:10.370859 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.371028 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.371092 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:35:10.377544 27081 raft_consensus.cc:1261] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Refusing update from remote peer 9c528a6718464a6e80337257e704ce4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:10.377548 27238 raft_consensus.cc:1261] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Refusing update from remote peer 9c528a6718464a6e80337257e704ce4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:10.378340 27484 consensus_queue.cc:1035] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:10.378551 27486 consensus_queue.cc:1035] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:10.388173 27491 mvcc.cc:205] Tried to move back new op lower bound from 7315625207298609152 to 7315625206941581312. Current Snapshot: MvccSnapshot[applied={T|T < 7315625207298609152}]
I20260806 18:35:10.397961 26909 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:10.398165 26909 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:10.400930 26909 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760: Sending DeleteTablet for 3 replicas of tablet 2579bf64fa494a5cb342906ff318daa2
I20260806 18:35:10.401804 27059 tablet_service.cc:1514] Processing DeleteTablet for tablet 2579bf64fa494a5cb342906ff318daa2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:10 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:50650
I20260806 18:35:10.402107 27369 tablet_service.cc:1514] Processing DeleteTablet for tablet 2579bf64fa494a5cb342906ff318daa2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:10 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:48502
I20260806 18:35:10.402118 27216 tablet_service.cc:1514] Processing DeleteTablet for tablet 2579bf64fa494a5cb342906ff318daa2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:10 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:45150
I20260806 18:35:10.402496 27500 tablet_replica.cc:330] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: stopping tablet replica
I20260806 18:35:10.402683 27501 tablet_replica.cc:330] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: stopping tablet replica
I20260806 18:35:10.402900 27500 raft_consensus.cc:2229] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:10.402940 27502 tablet_replica.cc:330] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: stopping tablet replica
I20260806 18:35:10.403200 27502 raft_consensus.cc:2229] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:35:10.403360 27500 raft_consensus.cc:2258] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:10.403589 27501 raft_consensus.cc:2229] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:10.403506 26909 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:35:10.403980 26909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:35:10.404417 27501 raft_consensus.cc:2258] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:10.404104 27502 raft_consensus.cc:2258] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:10.405238 27500 ts_tablet_manager.cc:1905] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:10.408830 27500 ts_tablet_manager.cc:1918] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:10.408972 27500 log.cc:1196] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/wal/wals/2579bf64fa494a5cb342906ff318daa2
I20260806 18:35:10.409453 27500 ts_tablet_manager.cc:1939] T 2579bf64fa494a5cb342906ff318daa2 P e18e0819d2cc4a548af63bfb501bbab8: Deleting consensus metadata
I20260806 18:35:10.410894 26897 catalog_manager.cc:4841] TS e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129:44399): tablet 2579bf64fa494a5cb342906ff318daa2 (table test-table [id=b79ae5468f5f418a873576f6a42dd614]) successfully deleted
I20260806 18:35:10.412209 27059 tablet_service.cc:1467] Processing CreateTablet for tablet 00014a598850415d8d53e22e22143e01 (DEFAULT_TABLE table=test-table [id=910a28e9d9cb4d8ba3d2acee4e4856b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:10.412532 27059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00014a598850415d8d53e22e22143e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.412488 27216 tablet_service.cc:1467] Processing CreateTablet for tablet 00014a598850415d8d53e22e22143e01 (DEFAULT_TABLE table=test-table [id=910a28e9d9cb4d8ba3d2acee4e4856b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:10.412765 27216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00014a598850415d8d53e22e22143e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.415324 27476 tablet_bootstrap.cc:492] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8: Bootstrap starting.
I20260806 18:35:10.415396 27478 tablet_bootstrap.cc:492] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6: Bootstrap starting.
I20260806 18:35:10.411877 27502 ts_tablet_manager.cc:1905] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:10.416527 27478 tablet_bootstrap.cc:654] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.416527 27476 tablet_bootstrap.cc:654] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.412467 27369 tablet_service.cc:1467] Processing CreateTablet for tablet 00014a598850415d8d53e22e22143e01 (DEFAULT_TABLE table=test-table [id=910a28e9d9cb4d8ba3d2acee4e4856b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:10.418488 27369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00014a598850415d8d53e22e22143e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.418684 27502 ts_tablet_manager.cc:1918] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:10.418792 27502 log.cc:1196] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/2579bf64fa494a5cb342906ff318daa2
I20260806 18:35:10.419134 27502 ts_tablet_manager.cc:1939] T 2579bf64fa494a5cb342906ff318daa2 P 9c528a6718464a6e80337257e704ce4e: Deleting consensus metadata
I20260806 18:35:10.419117 27501 ts_tablet_manager.cc:1905] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:10.421036 27477 tablet_bootstrap.cc:492] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e: Bootstrap starting.
I20260806 18:35:10.421396 26898 catalog_manager.cc:4841] TS 9c528a6718464a6e80337257e704ce4e (127.18.125.131:44057): tablet 2579bf64fa494a5cb342906ff318daa2 (table test-table [id=b79ae5468f5f418a873576f6a42dd614]) successfully deleted
I20260806 18:35:10.422223 27477 tablet_bootstrap.cc:654] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.422709 27501 ts_tablet_manager.cc:1918] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:10.422818 27501 log.cc:1196] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/2579bf64fa494a5cb342906ff318daa2
I20260806 18:35:10.423158 27501 ts_tablet_manager.cc:1939] T 2579bf64fa494a5cb342906ff318daa2 P f1ff8761a16d4560b2688ff332e3b6b6: Deleting consensus metadata
I20260806 18:35:10.423470 27476 tablet_bootstrap.cc:492] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8: No bootstrap required, opened a new log
I20260806 18:35:10.423589 27476 ts_tablet_manager.cc:1397] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260806 18:35:10.424290 27476 raft_consensus.cc:348] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.424453 27476 raft_consensus.cc:374] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.424495 27476 raft_consensus.cc:729] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18e0819d2cc4a548af63bfb501bbab8, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.424638 27476 consensus_queue.cc:260] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.424638 26898 catalog_manager.cc:4841] TS f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141): tablet 2579bf64fa494a5cb342906ff318daa2 (table test-table [id=b79ae5468f5f418a873576f6a42dd614]) successfully deleted
I20260806 18:35:10.424942 27476 ts_tablet_manager.cc:1428] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:35:10.425248 27478 tablet_bootstrap.cc:492] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6: No bootstrap required, opened a new log
I20260806 18:35:10.425248 27477 tablet_bootstrap.cc:492] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e: No bootstrap required, opened a new log
I20260806 18:35:10.425329 27478 ts_tablet_manager.cc:1397] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260806 18:35:10.425329 27477 ts_tablet_manager.cc:1397] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 18:35:10.425796 27477 raft_consensus.cc:348] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.425796 27478 raft_consensus.cc:348] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.425907 27477 raft_consensus.cc:374] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.425913 27478 raft_consensus.cc:374] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.425946 27478 raft_consensus.cc:729] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ff8761a16d4560b2688ff332e3b6b6, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.425946 27477 raft_consensus.cc:729] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c528a6718464a6e80337257e704ce4e, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.426075 27477 consensus_queue.cc:260] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.426075 27478 consensus_queue.cc:260] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.426323 27478 ts_tablet_manager.cc:1428] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:35:10.426323 27477 ts_tablet_manager.cc:1428] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:35:10.519955 27496 raft_consensus.cc:482] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:35:10.520155 27496 raft_consensus.cc:504] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.521159 27081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00014a598850415d8d53e22e22143e01" candidate_uuid: "9c528a6718464a6e80337257e704ce4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18e0819d2cc4a548af63bfb501bbab8" is_pre_election: true
I20260806 18:35:10.521338 27081 raft_consensus.cc:2454] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c528a6718464a6e80337257e704ce4e in term 0.
I20260806 18:35:10.521682 27496 leader_election.cc:288] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129:44399), f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141)
I20260806 18:35:10.521741 27319 leader_election.cc:302] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c528a6718464a6e80337257e704ce4e, e18e0819d2cc4a548af63bfb501bbab8; no voters:
I20260806 18:35:10.521991 27486 raft_consensus.cc:2781] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:35:10.522055 27486 raft_consensus.cc:482] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:35:10.522013 27238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00014a598850415d8d53e22e22143e01" candidate_uuid: "9c528a6718464a6e80337257e704ce4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" is_pre_election: true
I20260806 18:35:10.522119 27486 raft_consensus.cc:3037] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.522166 27238 raft_consensus.cc:2454] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c528a6718464a6e80337257e704ce4e in term 0.
I20260806 18:35:10.523455 27486 raft_consensus.cc:504] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.523867 27486 leader_election.cc:288] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [CANDIDATE]: Term 1 election: Requested vote from peers e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129:44399), f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141)
I20260806 18:35:10.524262 27081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00014a598850415d8d53e22e22143e01" candidate_uuid: "9c528a6718464a6e80337257e704ce4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18e0819d2cc4a548af63bfb501bbab8"
I20260806 18:35:10.524325 27238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00014a598850415d8d53e22e22143e01" candidate_uuid: "9c528a6718464a6e80337257e704ce4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ff8761a16d4560b2688ff332e3b6b6"
I20260806 18:35:10.524420 27081 raft_consensus.cc:3037] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.524490 27238 raft_consensus.cc:3037] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.525429 27238 raft_consensus.cc:2454] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c528a6718464a6e80337257e704ce4e in term 1.
I20260806 18:35:10.525671 27081 raft_consensus.cc:2454] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c528a6718464a6e80337257e704ce4e in term 1.
I20260806 18:35:10.525781 27320 leader_election.cc:302] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c528a6718464a6e80337257e704ce4e, f1ff8761a16d4560b2688ff332e3b6b6; no voters:
I20260806 18:35:10.525986 27486 raft_consensus.cc:2781] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:10.526082 27486 raft_consensus.cc:686] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 1 LEADER]: Becoming Leader. State: Replica: 9c528a6718464a6e80337257e704ce4e, State: Running, Role: LEADER
I20260806 18:35:10.526240 27486 consensus_queue.cc:237] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.528187 26910 catalog_manager.cc:5495] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c528a6718464a6e80337257e704ce4e (127.18.125.131). New cstate: current_term: 1 leader_uuid: "9c528a6718464a6e80337257e704ce4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } health_report { overall_health: HEALTHY } } }
Aug 06 18:35:10 dist-test-slave-8w9v krb5kdc[26865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041308, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.190@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
I20260806 18:35:10.583163 27512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.574023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:41669 (local address 127.0.0.1:36266)
0806 18:35:10.574367 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:10.574382 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:10.574485 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:10.574729 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:10.574731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:10.575140 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:10.575239 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.575246 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.576068 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.576071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.576678 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.576687 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.577338 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.577341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.577412 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.578010 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:10.578022 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:10.579569 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:10.581417 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:10.581428 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:10.581434 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:10.581680 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:10.582001 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:10.582003 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:10.582004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:10.582082 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:10.582620 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:10.582632 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:10.582740 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 18:35:10.582838 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 18:35:10.583019 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":117,"threads_started":1}
I20260806 18:35:10.583316 27514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.574125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36266 (local address 127.18.125.190:41669)
0806 18:35:10.574473 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.574482 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.574500 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:35:10.574565 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.574568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.574611 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.574688 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:10.575376 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.575948 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.576816 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.577207 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.577308 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.579733 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:10.579736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:10.579738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:10.579760 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:10.581263 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:10.581816 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:10.581820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:10.581821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:10.581906 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:10.582164 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:10.582166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:10.582167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:10.582372 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:10.582452 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.582911 (+ 459us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.583096 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":271,"thread_start_us":138,"threads_started":1}
I20260806 18:35:10.587654 26910 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:36266:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:35:10.588574 26910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:35:10.593866 27216 tablet_service.cc:1467] Processing CreateTablet for tablet 7222b2b06b944310bb34147ec7e09944 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bb00b9d37d74bb1921313623ff8d590]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:35:10.593866 27059 tablet_service.cc:1467] Processing CreateTablet for tablet 7222b2b06b944310bb34147ec7e09944 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bb00b9d37d74bb1921313623ff8d590]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:35:10.593868 27369 tablet_service.cc:1467] Processing CreateTablet for tablet 7222b2b06b944310bb34147ec7e09944 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bb00b9d37d74bb1921313623ff8d590]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:35:10.594352 27369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7222b2b06b944310bb34147ec7e09944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.594448 27216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7222b2b06b944310bb34147ec7e09944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.594460 27059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7222b2b06b944310bb34147ec7e09944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.597249 27477 tablet_bootstrap.cc:492] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e: Bootstrap starting.
I20260806 18:35:10.597506 27478 tablet_bootstrap.cc:492] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: Bootstrap starting.
I20260806 18:35:10.598076 27476 tablet_bootstrap.cc:492] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8: Bootstrap starting.
I20260806 18:35:10.598239 27477 tablet_bootstrap.cc:654] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.598575 27478 tablet_bootstrap.cc:654] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.599123 27476 tablet_bootstrap.cc:654] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.600270 27478 tablet_bootstrap.cc:492] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: No bootstrap required, opened a new log
I20260806 18:35:10.600412 27478 ts_tablet_manager.cc:1397] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:10.600462 27476 tablet_bootstrap.cc:492] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8: No bootstrap required, opened a new log
I20260806 18:35:10.600517 27477 tablet_bootstrap.cc:492] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e: No bootstrap required, opened a new log
I20260806 18:35:10.600555 27476 ts_tablet_manager.cc:1397] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:10.600615 27477 ts_tablet_manager.cc:1397] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:10.600903 27478 raft_consensus.cc:348] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.601038 27478 raft_consensus.cc:374] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.601073 27478 raft_consensus.cc:729] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ff8761a16d4560b2688ff332e3b6b6, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.601006 27476 raft_consensus.cc:348] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.600996 27477 raft_consensus.cc:348] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.601119 27476 raft_consensus.cc:374] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.601132 27477 raft_consensus.cc:374] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.601150 27476 raft_consensus.cc:729] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18e0819d2cc4a548af63bfb501bbab8, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.601176 27477 raft_consensus.cc:729] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c528a6718464a6e80337257e704ce4e, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.601209 27478 consensus_queue.cc:260] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.601260 27476 consensus_queue.cc:260] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.601333 27477 consensus_queue.cc:260] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.601513 27478 ts_tablet_manager.cc:1428] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:35:10.601521 27476 ts_tablet_manager.cc:1428] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:35:10.601648 27477 ts_tablet_manager.cc:1428] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:35:10.601588 27486 tablet_replica.cc:439] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } } }
I20260806 18:35:10.601722 27486 tablet_replica.cc:442] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:35:10.601738 27482 tablet_replica.cc:439] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } } }
I20260806 18:35:10.601851 27482 tablet_replica.cc:442] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:35:10.601969 27483 tablet_replica.cc:439] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } } }
I20260806 18:35:10.602121 27483 tablet_replica.cc:442] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:35:10.630300 27483 raft_consensus.cc:482] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:35:10.630455 27483 raft_consensus.cc:504] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.631753 27483 leader_election.cc:288] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129:44399), 9c528a6718464a6e80337257e704ce4e (127.18.125.131:44057)
I20260806 18:35:10.637593 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.632247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:56995 (local address 127.18.125.131:44057)
0806 18:35:10.632387 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.632394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.632597 (+ 203us) server_negotiation.cc:407] Connection header received
0806 18:35:10.632779 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.632782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.632838 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.632889 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:10.633296 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.634035 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.636084 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.636768 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.636913 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.636970 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.637366 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.637465 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:35:10.637593 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.632158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:38149 (local address 127.18.125.129:44399)
0806 18:35:10.632341 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.632348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.632531 (+ 183us) server_negotiation.cc:407] Connection header received
0806 18:35:10.632745 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.632750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.632802 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.632888 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:10.633211 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.634027 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.636084 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.636773 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.636913 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.636970 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.637366 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.637459 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:35:10.637609 27519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.632101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:44057 (local address 127.18.125.130:56995)
0806 18:35:10.632492 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:10.632503 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:10.632631 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:10.633042 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:10.633045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:10.633060 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:10.633175 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.633181 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.634212 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.634215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.635885 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.635899 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.636913 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.636917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.637167 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.637211 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 18:35:10.637296 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:35:10.637423 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":267,"thread_start_us":114,"threads_started":1}
I20260806 18:35:10.637614 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.631983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44399 (local address 127.18.125.130:38149)
0806 18:35:10.632431 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:10.632443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:10.632605 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:10.632920 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:10.632923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:10.632946 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:10.633068 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.633075 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.634194 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.634200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.635893 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.635899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.636913 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.636917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.637162 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.637209 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 18:35:10.637296 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 18:35:10.637423 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":247,"thread_start_us":156,"threads_started":1}
I20260806 18:35:10.638185 27370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7222b2b06b944310bb34147ec7e09944" candidate_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c528a6718464a6e80337257e704ce4e" is_pre_election: true
I20260806 18:35:10.638403 27081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7222b2b06b944310bb34147ec7e09944" candidate_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18e0819d2cc4a548af63bfb501bbab8" is_pre_election: true
I20260806 18:35:10.638526 27370 raft_consensus.cc:2454] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ff8761a16d4560b2688ff332e3b6b6 in term 0.
I20260806 18:35:10.638577 27081 raft_consensus.cc:2454] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ff8761a16d4560b2688ff332e3b6b6 in term 0.
I20260806 18:35:10.638996 27165 leader_election.cc:302] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9c528a6718464a6e80337257e704ce4e, e18e0819d2cc4a548af63bfb501bbab8, f1ff8761a16d4560b2688ff332e3b6b6; no voters:
I20260806 18:35:10.639345 27483 raft_consensus.cc:2781] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:35:10.639408 27483 raft_consensus.cc:482] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:35:10.639436 27483 raft_consensus.cc:3037] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.640576 27483 raft_consensus.cc:504] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.640965 27483 leader_election.cc:288] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [CANDIDATE]: Term 1 election: Requested vote from peers e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129:44399), 9c528a6718464a6e80337257e704ce4e (127.18.125.131:44057)
I20260806 18:35:10.641372 27081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7222b2b06b944310bb34147ec7e09944" candidate_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18e0819d2cc4a548af63bfb501bbab8"
I20260806 18:35:10.641381 27370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7222b2b06b944310bb34147ec7e09944" candidate_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c528a6718464a6e80337257e704ce4e"
I20260806 18:35:10.641503 27081 raft_consensus.cc:3037] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.641503 27370 raft_consensus.cc:3037] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.642656 27081 raft_consensus.cc:2454] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ff8761a16d4560b2688ff332e3b6b6 in term 1.
I20260806 18:35:10.642781 27370 raft_consensus.cc:2454] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ff8761a16d4560b2688ff332e3b6b6 in term 1.
I20260806 18:35:10.643087 27165 leader_election.cc:302] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e18e0819d2cc4a548af63bfb501bbab8, f1ff8761a16d4560b2688ff332e3b6b6; no voters:
I20260806 18:35:10.643349 27483 raft_consensus.cc:2781] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:10.643632 27483 raft_consensus.cc:686] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 LEADER]: Becoming Leader. State: Replica: f1ff8761a16d4560b2688ff332e3b6b6, State: Running, Role: LEADER
I20260806 18:35:10.643936 27483 consensus_queue.cc:237] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.645695 27521 tablet_replica.cc:439] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: TxnStatusTablet state changed. Reason: New leader f1ff8761a16d4560b2688ff332e3b6b6. Latest consensus state: current_term: 1 leader_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } } }
I20260806 18:35:10.645813 27521 tablet_replica.cc:442] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:35:10.646092 27523 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:35:10.646311 27523 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:35:10.646843 26909 catalog_manager.cc:5495] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130). New cstate: current_term: 1 leader_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } health_report { overall_health: UNKNOWN } } }
I20260806 18:35:10.666990 27512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.662999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:43141 (local address 127.0.0.1:45162)
0806 18:35:10.663266 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:10.663284 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:10.663399 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:10.663726 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:10.663729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:10.663737 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:35:10.663854 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.663860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.664723 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.664726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.665344 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.665353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.665824 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.665827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.665936 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.665982 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:35:10.666678 (+ 696us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:35:10.666696 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:35:10.666768 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:35:10.666857 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260806 18:35:10.666990 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.663144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45162 (local address 127.18.125.130:43141)
0806 18:35:10.663284 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.663289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.663337 (+ 48us) server_negotiation.cc:407] Connection header received
0806 18:35:10.663543 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.663545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.663603 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.663688 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:10.663961 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.664600 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.665478 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.665704 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.665819 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.666144 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:10.666572 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:10.666702 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.666779 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.666837 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:35:10.670373 27081 raft_consensus.cc:1261] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Refusing update from remote peer f1ff8761a16d4560b2688ff332e3b6b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:10.670650 27370 raft_consensus.cc:1261] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Refusing update from remote peer f1ff8761a16d4560b2688ff332e3b6b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:10.670964 27521 consensus_queue.cc:1035] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:10.671254 27483 consensus_queue.cc:1035] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:10.673192 27482 tablet_replica.cc:439] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8: TxnStatusTablet state changed. Reason: New leader f1ff8761a16d4560b2688ff332e3b6b6. Latest consensus state: current_term: 1 leader_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } } }
I20260806 18:35:10.673341 27482 tablet_replica.cc:442] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:35:10.675011 27486 tablet_replica.cc:439] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e: TxnStatusTablet state changed. Reason: New leader f1ff8761a16d4560b2688ff332e3b6b6. Latest consensus state: current_term: 1 leader_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } } }
I20260806 18:35:10.675160 27486 tablet_replica.cc:442] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:35:10.677008 27521 tablet_replica.cc:439] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } } }
I20260806 18:35:10.677145 27521 tablet_replica.cc:442] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:35:10.677309 27490 mvcc.cc:205] Tried to move back new op lower bound from 7315625208500129792 to 7315625208400711680. Current Snapshot: MvccSnapshot[applied={T|T < 7315625208500129792}]
I20260806 18:35:10.677511 27482 tablet_replica.cc:439] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } } }
I20260806 18:35:10.677618 27482 tablet_replica.cc:442] T 7222b2b06b944310bb34147ec7e09944 P e18e0819d2cc4a548af63bfb501bbab8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:35:10.678732 27528 tablet_replica.cc:439] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } } }
I20260806 18:35:10.678947 27528 tablet_replica.cc:442] T 7222b2b06b944310bb34147ec7e09944 P f1ff8761a16d4560b2688ff332e3b6b6: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:35:10.680521 27486 tablet_replica.cc:439] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } } }
I20260806 18:35:10.680651 27486 tablet_replica.cc:442] T 7222b2b06b944310bb34147ec7e09944 P 9c528a6718464a6e80337257e704ce4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:35:10.680733 27489 mvcc.cc:205] Tried to move back new op lower bound from 7315625208500129792 to 7315625208400711680. Current Snapshot: MvccSnapshot[applied={T|T < 7315625208500129792 or (T in {7315625208500129792})}]
I20260806 18:35:10.692381 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.688921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:33995 (local address 127.18.125.130:43141)
0806 18:35:10.689083 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.689088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.689225 (+ 137us) server_negotiation.cc:407] Connection header received
0806 18:35:10.689406 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.689409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.689468 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.689552 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:10.689818 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.690338 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.691219 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.691469 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.691582 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.691883 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:10.691969 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:10.692052 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.692174 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.692228 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 18:35:10.692394 27531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.688778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:43141 (local address 127.18.125.131:33995)
0806 18:35:10.689138 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:10.689151 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:10.689294 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:10.689566 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:10.689569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:10.689577 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:35:10.689698 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.689703 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.690455 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.690459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.691092 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.691100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.691590 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.691594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.691689 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.691738 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:35:10.692059 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:35:10.692068 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 18:35:10.692154 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:35:10.692251 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":127,"threads_started":1}
I20260806 18:35:10.699815 27238 raft_consensus.cc:1261] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Refusing update from remote peer 9c528a6718464a6e80337257e704ce4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:10.699820 27081 raft_consensus.cc:1261] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Refusing update from remote peer 9c528a6718464a6e80337257e704ce4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:10.700282 27486 consensus_queue.cc:1035] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:10.700568 27486 consensus_queue.cc:1035] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:10.725011 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.720594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:47879 (local address 127.18.125.131:44057)
0806 18:35:10.720838 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.720843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.720920 (+ 77us) server_negotiation.cc:407] Connection header received
0806 18:35:10.721081 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.721084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.721137 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.721212 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:10.721638 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.722459 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.723756 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.723999 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.724100 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.724489 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:10.724581 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:10.724627 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.724783 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.724859 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260806 18:35:10.725481 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.720496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:44057 (local address 127.18.125.130:47879)
0806 18:35:10.720837 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:10.720850 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:10.720976 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:10.721358 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:10.721361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:10.721370 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:35:10.721504 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.721510 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.722601 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.722604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.723606 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.723617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.724158 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.724162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.724261 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.724321 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:35:10.724688 (+ 367us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:35:10.724694 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:35:10.725233 (+ 539us) client_negotiation.cc:240] Negotiation successful
0806 18:35:10.725323 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":111,"threads_started":1}
I20260806 18:35:10.746685 26911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:10.746871 26911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:10.749272 26911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760: Sending DeleteTablet for 3 replicas of tablet 00014a598850415d8d53e22e22143e01
I20260806 18:35:10.749966 27059 tablet_service.cc:1514] Processing DeleteTablet for tablet 00014a598850415d8d53e22e22143e01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:10 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:50650
I20260806 18:35:10.750183 27500 tablet_replica.cc:330] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8: stopping tablet replica
I20260806 18:35:10.750238 27369 tablet_service.cc:1514] Processing DeleteTablet for tablet 00014a598850415d8d53e22e22143e01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:10 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:48502
I20260806 18:35:10.750325 27500 raft_consensus.cc:2229] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:10.750443 27502 tablet_replica.cc:330] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e: stopping tablet replica
I20260806 18:35:10.750466 27500 raft_consensus.cc:2258] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:10.750607 27502 raft_consensus.cc:2229] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:35:10.750861 27502 raft_consensus.cc:2258] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:10.751366 27500 ts_tablet_manager.cc:1905] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:10.751729 27502 ts_tablet_manager.cc:1905] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:10.756892 27502 ts_tablet_manager.cc:1918] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:35:10.757228 27502 log.cc:1196] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/00014a598850415d8d53e22e22143e01
I20260806 18:35:10.757272 27216 tablet_service.cc:1514] Processing DeleteTablet for tablet 00014a598850415d8d53e22e22143e01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:10 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:45150
I20260806 18:35:10.757513 27501 tablet_replica.cc:330] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6: stopping tablet replica
I20260806 18:35:10.757592 27502 ts_tablet_manager.cc:1939] T 00014a598850415d8d53e22e22143e01 P 9c528a6718464a6e80337257e704ce4e: Deleting consensus metadata
I20260806 18:35:10.757643 27501 raft_consensus.cc:2229] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:10.757802 27501 raft_consensus.cc:2258] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:10.758873 27501 ts_tablet_manager.cc:1905] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:10.759109 26898 catalog_manager.cc:4841] TS 9c528a6718464a6e80337257e704ce4e (127.18.125.131:44057): tablet 00014a598850415d8d53e22e22143e01 (table test-table [id=910a28e9d9cb4d8ba3d2acee4e4856b7]) successfully deleted
I20260806 18:35:10.761433 27500 ts_tablet_manager.cc:1918] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:35:10.761551 27500 log.cc:1196] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/wal/wals/00014a598850415d8d53e22e22143e01
I20260806 18:35:10.761875 27500 ts_tablet_manager.cc:1939] T 00014a598850415d8d53e22e22143e01 P e18e0819d2cc4a548af63bfb501bbab8: Deleting consensus metadata
I20260806 18:35:10.762434 27501 ts_tablet_manager.cc:1918] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:35:10.762540 27501 log.cc:1196] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/00014a598850415d8d53e22e22143e01
I20260806 18:35:10.762892 26897 catalog_manager.cc:4841] TS e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129:44399): tablet 00014a598850415d8d53e22e22143e01 (table test-table [id=910a28e9d9cb4d8ba3d2acee4e4856b7]) successfully deleted
I20260806 18:35:10.763181 27501 ts_tablet_manager.cc:1939] T 00014a598850415d8d53e22e22143e01 P f1ff8761a16d4560b2688ff332e3b6b6: Deleting consensus metadata
I20260806 18:35:10.764261 26898 catalog_manager.cc:4841] TS f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141): tablet 00014a598850415d8d53e22e22143e01 (table test-table [id=910a28e9d9cb4d8ba3d2acee4e4856b7]) successfully deleted
Aug 06 18:35:10 dist-test-slave-8w9v krb5kdc[26865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:35:10 dist-test-slave-8w9v krb5kdc[26865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:10 dist-test-slave-8w9v krb5kdc[26865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:35:10.787529 27514 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:10.780078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36270 (local address 127.18.125.190:41669)
0806 18:35:10.780249 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.780254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.780653 (+ 399us) server_negotiation.cc:407] Connection header received
0806 18:35:10.780851 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.780855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.780903 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.780996 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:10.781758 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.782320 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.783217 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.783467 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.783585 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.787386 (+ 3801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36270: BlockingRecv error: recv got EOF from 127.0.0.1:36270 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 18:35:10 dist-test-slave-8w9v krb5kdc[26865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM
I20260806 18:35:10.798964 27514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.789396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36278 (local address 127.18.125.190:41669)
0806 18:35:10.789523 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.789527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.789738 (+ 211us) server_negotiation.cc:407] Connection header received
0806 18:35:10.789912 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.789915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.789970 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.790055 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:10.790678 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.791213 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.792045 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.792382 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.792494 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.795462 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:10.795465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:10.795467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:10.795495 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:10.797130 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:10.797652 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:10.797655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:10.797656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:10.797734 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:10.798075 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:10.798077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:10.798078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:10.798270 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:10.798377 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.798682 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.798791 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:35:10.802029 26911 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:35:10.802561 26911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:35:10.808396 27059 tablet_service.cc:1467] Processing CreateTablet for tablet cf2b39cfd31c4c8d8030737e8633d05a (DEFAULT_TABLE table=test-table [id=a91b887d02b04388a3f5da41b73ac17c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:10.808495 27216 tablet_service.cc:1467] Processing CreateTablet for tablet cf2b39cfd31c4c8d8030737e8633d05a (DEFAULT_TABLE table=test-table [id=a91b887d02b04388a3f5da41b73ac17c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:10.808480 27369 tablet_service.cc:1467] Processing CreateTablet for tablet cf2b39cfd31c4c8d8030737e8633d05a (DEFAULT_TABLE table=test-table [id=a91b887d02b04388a3f5da41b73ac17c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:10.808800 27216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2b39cfd31c4c8d8030737e8633d05a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.808804 27059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2b39cfd31c4c8d8030737e8633d05a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.808800 27369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2b39cfd31c4c8d8030737e8633d05a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.812006 27476 tablet_bootstrap.cc:492] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8: Bootstrap starting.
I20260806 18:35:10.813187 27477 tablet_bootstrap.cc:492] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e: Bootstrap starting.
I20260806 18:35:10.813270 27476 tablet_bootstrap.cc:654] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.814409 27477 tablet_bootstrap.cc:654] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.814741 27478 tablet_bootstrap.cc:492] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6: Bootstrap starting.
I20260806 18:35:10.816069 27477 tablet_bootstrap.cc:492] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e: No bootstrap required, opened a new log
I20260806 18:35:10.816190 27478 tablet_bootstrap.cc:654] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.816232 27477 ts_tablet_manager.cc:1397] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260806 18:35:10.816705 27477 raft_consensus.cc:348] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.816840 27477 raft_consensus.cc:374] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.816872 27477 raft_consensus.cc:729] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c528a6718464a6e80337257e704ce4e, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.817057 27477 consensus_queue.cc:260] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.817322 27476 tablet_bootstrap.cc:492] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8: No bootstrap required, opened a new log
I20260806 18:35:10.817442 27476 ts_tablet_manager.cc:1397] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260806 18:35:10.818003 27477 ts_tablet_manager.cc:1428] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 18:35:10.818362 27476 raft_consensus.cc:348] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.818758 27476 raft_consensus.cc:374] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.818842 27476 raft_consensus.cc:729] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18e0819d2cc4a548af63bfb501bbab8, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.819027 27476 consensus_queue.cc:260] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.819362 27476 ts_tablet_manager.cc:1428] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 18:35:10.821193 27478 tablet_bootstrap.cc:492] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6: No bootstrap required, opened a new log
I20260806 18:35:10.821650 27478 ts_tablet_manager.cc:1397] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260806 18:35:10.823753 27478 raft_consensus.cc:348] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.823916 27478 raft_consensus.cc:374] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.823976 27478 raft_consensus.cc:729] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ff8761a16d4560b2688ff332e3b6b6, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.824152 27478 consensus_queue.cc:260] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.824972 27478 ts_tablet_manager.cc:1428] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:10.852589 27482 raft_consensus.cc:482] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:35:10.852738 27482 raft_consensus.cc:504] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.853942 27482 leader_election.cc:288] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141), 9c528a6718464a6e80337257e704ce4e (127.18.125.131:44057)
I20260806 18:35:10.859015 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.854277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:43141 (local address 127.18.125.129:48629)
0806 18:35:10.854554 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:10.854572 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:10.854685 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:10.855086 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:10.855090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:10.855136 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:10.855273 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.855280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.856148 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.856151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.857374 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.857384 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.858561 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.858565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.858661 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.858688 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:35:10.858752 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:35:10.858835 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"spinlock_wait_cycles":34432,"thread_start_us":99,"threads_started":1}
I20260806 18:35:10.859004 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.854359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:48629 (local address 127.18.125.130:43141)
0806 18:35:10.854500 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.854504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.854654 (+ 150us) server_negotiation.cc:407] Connection header received
0806 18:35:10.854802 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.854805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.854864 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.854951 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:10.855410 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.856000 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.857984 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.858443 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.858575 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.858637 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.858786 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.858872 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:35:10.859694 27238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2b39cfd31c4c8d8030737e8633d05a" candidate_uuid: "e18e0819d2cc4a548af63bfb501bbab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" is_pre_election: true
I20260806 18:35:10.859863 27238 raft_consensus.cc:2454] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e18e0819d2cc4a548af63bfb501bbab8 in term 0.
I20260806 18:35:10.859915 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.855113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:59311 (local address 127.18.125.131:44057)
0806 18:35:10.855248 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.855253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.855267 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:35:10.855321 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.855323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.855395 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.855474 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:10.856204 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.856960 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.858500 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.859116 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.859250 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.859316 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.859677 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.859782 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:35:10.860394 27557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.854689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:44057 (local address 127.18.125.129:59311)
0806 18:35:10.855007 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:10.855022 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:10.855144 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:10.855493 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:10.855496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:10.855511 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:10.856026 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.856034 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.857096 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.857099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.858344 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:10.858356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.859410 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.859414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:10.859520 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.859556 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 18:35:10.860151 (+ 595us) client_negotiation.cc:240] Negotiation successful
0806 18:35:10.860242 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"spinlock_wait_cycles":1792,"thread_start_us":128,"threads_started":1}
I20260806 18:35:10.860327 27007 leader_election.cc:302] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e18e0819d2cc4a548af63bfb501bbab8, f1ff8761a16d4560b2688ff332e3b6b6; no voters:
I20260806 18:35:10.860731 27482 raft_consensus.cc:2781] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:35:10.860788 27482 raft_consensus.cc:482] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:35:10.860832 27482 raft_consensus.cc:3037] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.860937 27393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2b39cfd31c4c8d8030737e8633d05a" candidate_uuid: "e18e0819d2cc4a548af63bfb501bbab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c528a6718464a6e80337257e704ce4e" is_pre_election: true
I20260806 18:35:10.861090 27393 raft_consensus.cc:2454] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e18e0819d2cc4a548af63bfb501bbab8 in term 0.
I20260806 18:35:10.862115 27482 raft_consensus.cc:504] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.862494 27482 leader_election.cc:288] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [CANDIDATE]: Term 1 election: Requested vote from peers f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141), 9c528a6718464a6e80337257e704ce4e (127.18.125.131:44057)
I20260806 18:35:10.862960 27238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2b39cfd31c4c8d8030737e8633d05a" candidate_uuid: "e18e0819d2cc4a548af63bfb501bbab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ff8761a16d4560b2688ff332e3b6b6"
I20260806 18:35:10.863039 27393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2b39cfd31c4c8d8030737e8633d05a" candidate_uuid: "e18e0819d2cc4a548af63bfb501bbab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c528a6718464a6e80337257e704ce4e"
I20260806 18:35:10.863103 27238 raft_consensus.cc:3037] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.863147 27393 raft_consensus.cc:3037] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:10.864353 27238 raft_consensus.cc:2454] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e18e0819d2cc4a548af63bfb501bbab8 in term 1.
I20260806 18:35:10.864382 27393 raft_consensus.cc:2454] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e18e0819d2cc4a548af63bfb501bbab8 in term 1.
I20260806 18:35:10.864704 27007 leader_election.cc:302] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c528a6718464a6e80337257e704ce4e, e18e0819d2cc4a548af63bfb501bbab8; no voters:
I20260806 18:35:10.864898 27482 raft_consensus.cc:2781] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:10.865177 27482 raft_consensus.cc:686] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 1 LEADER]: Becoming Leader. State: Replica: e18e0819d2cc4a548af63bfb501bbab8, State: Running, Role: LEADER
I20260806 18:35:10.865460 27482 consensus_queue.cc:237] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.868180 26910 catalog_manager.cc:5495] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129). New cstate: current_term: 1 leader_uuid: "e18e0819d2cc4a548af63bfb501bbab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } health_report { overall_health: UNKNOWN } } }
I20260806 18:35:10.879537 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:10.876069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50656 (local address 127.18.125.129:44399)
0806 18:35:10.876237 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:10.876240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:10.876253 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:35:10.876354 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:10.876357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:10.876415 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:10.876517 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:10.876720 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.877219 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.878111 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:10.878359 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:10.878442 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:10.878688 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:10.879173 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:10.879263 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:10.879359 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 18:35:10.879416 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 18:35:10.882603 27238 raft_consensus.cc:1261] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Refusing update from remote peer e18e0819d2cc4a548af63bfb501bbab8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:10.882651 27393 raft_consensus.cc:1261] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Refusing update from remote peer e18e0819d2cc4a548af63bfb501bbab8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:10.883219 27482 consensus_queue.cc:1035] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:10.883427 27558 consensus_queue.cc:1035] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:10.897001 26910 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:10.897182 26910 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:10.899399 26910 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760: Sending DeleteTablet for 3 replicas of tablet cf2b39cfd31c4c8d8030737e8633d05a
I20260806 18:35:10.900079 27059 tablet_service.cc:1514] Processing DeleteTablet for tablet cf2b39cfd31c4c8d8030737e8633d05a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:10 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:50650
I20260806 18:35:10.900163 27369 tablet_service.cc:1514] Processing DeleteTablet for tablet cf2b39cfd31c4c8d8030737e8633d05a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:10 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:48502
I20260806 18:35:10.900225 27216 tablet_service.cc:1514] Processing DeleteTablet for tablet cf2b39cfd31c4c8d8030737e8633d05a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:10 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:45150
I20260806 18:35:10.900384 27500 tablet_replica.cc:330] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8: stopping tablet replica
I20260806 18:35:10.900502 27501 tablet_replica.cc:330] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6: stopping tablet replica
I20260806 18:35:10.900528 27500 raft_consensus.cc:2229] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:35:10.900662 27501 raft_consensus.cc:2229] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:10.900840 27501 raft_consensus.cc:2258] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:10.900840 27500 raft_consensus.cc:2258] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:10.901053 27502 tablet_replica.cc:330] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e: stopping tablet replica
I20260806 18:35:10.901191 27502 raft_consensus.cc:2229] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:10.901335 27502 raft_consensus.cc:2258] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:10.902143 27501 ts_tablet_manager.cc:1905] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:10.902498 26910 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:35:10.902940 26910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:35:10.904148 27500 ts_tablet_manager.cc:1905] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:10.904559 27501 ts_tablet_manager.cc:1918] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:10.904660 27501 log.cc:1196] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/cf2b39cfd31c4c8d8030737e8633d05a
I20260806 18:35:10.904906 27501 ts_tablet_manager.cc:1939] T cf2b39cfd31c4c8d8030737e8633d05a P f1ff8761a16d4560b2688ff332e3b6b6: Deleting consensus metadata
I20260806 18:35:10.905656 27502 ts_tablet_manager.cc:1905] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:10.905774 26898 catalog_manager.cc:4841] TS f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141): tablet cf2b39cfd31c4c8d8030737e8633d05a (table test-table [id=a91b887d02b04388a3f5da41b73ac17c]) successfully deleted
I20260806 18:35:10.906734 27500 ts_tablet_manager.cc:1918] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:10.906826 27500 log.cc:1196] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-0/wal/wals/cf2b39cfd31c4c8d8030737e8633d05a
I20260806 18:35:10.907078 27500 ts_tablet_manager.cc:1939] T cf2b39cfd31c4c8d8030737e8633d05a P e18e0819d2cc4a548af63bfb501bbab8: Deleting consensus metadata
I20260806 18:35:10.907886 26897 catalog_manager.cc:4841] TS e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129:44399): tablet cf2b39cfd31c4c8d8030737e8633d05a (table test-table [id=a91b887d02b04388a3f5da41b73ac17c]) successfully deleted
I20260806 18:35:10.907949 27502 ts_tablet_manager.cc:1918] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:10.908020 27502 log.cc:1196] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/cf2b39cfd31c4c8d8030737e8633d05a
I20260806 18:35:10.908305 27502 ts_tablet_manager.cc:1939] T cf2b39cfd31c4c8d8030737e8633d05a P 9c528a6718464a6e80337257e704ce4e: Deleting consensus metadata
I20260806 18:35:10.909041 26898 catalog_manager.cc:4841] TS 9c528a6718464a6e80337257e704ce4e (127.18.125.131:44057): tablet cf2b39cfd31c4c8d8030737e8633d05a (table test-table [id=a91b887d02b04388a3f5da41b73ac17c]) successfully deleted
I20260806 18:35:10.912374 27059 tablet_service.cc:1467] Processing CreateTablet for tablet 105a267d54d046e3a31b9ebb0196a236 (DEFAULT_TABLE table=test-table [id=b109035a3c0f493b964e30afc1bbb295]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:10.912467 27369 tablet_service.cc:1467] Processing CreateTablet for tablet 105a267d54d046e3a31b9ebb0196a236 (DEFAULT_TABLE table=test-table [id=b109035a3c0f493b964e30afc1bbb295]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:10.912467 27216 tablet_service.cc:1467] Processing CreateTablet for tablet 105a267d54d046e3a31b9ebb0196a236 (DEFAULT_TABLE table=test-table [id=b109035a3c0f493b964e30afc1bbb295]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:10.912680 27059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105a267d54d046e3a31b9ebb0196a236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.912738 27369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105a267d54d046e3a31b9ebb0196a236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.912738 27216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105a267d54d046e3a31b9ebb0196a236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:10.914690 27476 tablet_bootstrap.cc:492] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8: Bootstrap starting.
I20260806 18:35:10.915132 27477 tablet_bootstrap.cc:492] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e: Bootstrap starting.
I20260806 18:35:10.915132 27478 tablet_bootstrap.cc:492] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6: Bootstrap starting.
I20260806 18:35:10.915773 27476 tablet_bootstrap.cc:654] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.915921 27477 tablet_bootstrap.cc:654] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.915930 27478 tablet_bootstrap.cc:654] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:10.917158 27476 tablet_bootstrap.cc:492] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8: No bootstrap required, opened a new log
I20260806 18:35:10.917220 27477 tablet_bootstrap.cc:492] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e: No bootstrap required, opened a new log
I20260806 18:35:10.917246 27476 ts_tablet_manager.cc:1397] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:10.917305 27477 ts_tablet_manager.cc:1397] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 18:35:10.917495 27478 tablet_bootstrap.cc:492] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6: No bootstrap required, opened a new log
I20260806 18:35:10.917572 27478 ts_tablet_manager.cc:1397] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:35:10.917730 27477 raft_consensus.cc:348] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.917730 27476 raft_consensus.cc:348] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.917840 27477 raft_consensus.cc:374] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.917850 27476 raft_consensus.cc:374] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.917874 27477 raft_consensus.cc:729] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c528a6718464a6e80337257e704ce4e, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.917882 27476 raft_consensus.cc:729] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18e0819d2cc4a548af63bfb501bbab8, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.917982 27478 raft_consensus.cc:348] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.918005 27477 consensus_queue.cc:260] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.918092 27478 raft_consensus.cc:374] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:10.918125 27478 raft_consensus.cc:729] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ff8761a16d4560b2688ff332e3b6b6, State: Initialized, Role: FOLLOWER
I20260806 18:35:10.918237 27478 consensus_queue.cc:260] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.918292 27477 ts_tablet_manager.cc:1428] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:35:10.918459 27476 consensus_queue.cc:260] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:10.918514 27478 ts_tablet_manager.cc:1428] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:35:10.918740 27476 ts_tablet_manager.cc:1428] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:35:11.087865 27558 raft_consensus.cc:482] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:35:11.088044 27558 raft_consensus.cc:504] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:11.088580 27558 leader_election.cc:288] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141), 9c528a6718464a6e80337257e704ce4e (127.18.125.131:44057)
I20260806 18:35:11.089095 27238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105a267d54d046e3a31b9ebb0196a236" candidate_uuid: "e18e0819d2cc4a548af63bfb501bbab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" is_pre_election: true
I20260806 18:35:11.089130 27393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105a267d54d046e3a31b9ebb0196a236" candidate_uuid: "e18e0819d2cc4a548af63bfb501bbab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c528a6718464a6e80337257e704ce4e" is_pre_election: true
I20260806 18:35:11.089286 27238 raft_consensus.cc:2454] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e18e0819d2cc4a548af63bfb501bbab8 in term 0.
I20260806 18:35:11.089289 27393 raft_consensus.cc:2454] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e18e0819d2cc4a548af63bfb501bbab8 in term 0.
I20260806 18:35:11.089685 27007 leader_election.cc:302] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e18e0819d2cc4a548af63bfb501bbab8, f1ff8761a16d4560b2688ff332e3b6b6; no voters:
I20260806 18:35:11.089905 27558 raft_consensus.cc:2781] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:35:11.089986 27558 raft_consensus.cc:482] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:35:11.090041 27558 raft_consensus.cc:3037] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:11.091166 27558 raft_consensus.cc:504] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:11.091543 27558 leader_election.cc:288] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [CANDIDATE]: Term 1 election: Requested vote from peers f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141), 9c528a6718464a6e80337257e704ce4e (127.18.125.131:44057)
I20260806 18:35:11.091984 27238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105a267d54d046e3a31b9ebb0196a236" candidate_uuid: "e18e0819d2cc4a548af63bfb501bbab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ff8761a16d4560b2688ff332e3b6b6"
I20260806 18:35:11.092064 27393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105a267d54d046e3a31b9ebb0196a236" candidate_uuid: "e18e0819d2cc4a548af63bfb501bbab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c528a6718464a6e80337257e704ce4e"
I20260806 18:35:11.092180 27238 raft_consensus.cc:3037] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:11.092242 27393 raft_consensus.cc:3037] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:11.093204 27238 raft_consensus.cc:2454] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e18e0819d2cc4a548af63bfb501bbab8 in term 1.
I20260806 18:35:11.093293 27393 raft_consensus.cc:2454] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e18e0819d2cc4a548af63bfb501bbab8 in term 1.
I20260806 18:35:11.093662 27007 leader_election.cc:302] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c528a6718464a6e80337257e704ce4e, e18e0819d2cc4a548af63bfb501bbab8; no voters:
I20260806 18:35:11.093881 27558 raft_consensus.cc:2781] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:11.093992 27558 raft_consensus.cc:686] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 LEADER]: Becoming Leader. State: Replica: e18e0819d2cc4a548af63bfb501bbab8, State: Running, Role: LEADER
I20260806 18:35:11.094147 27558 consensus_queue.cc:237] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } }
I20260806 18:35:11.096107 26909 catalog_manager.cc:5495] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129). New cstate: current_term: 1 leader_uuid: "e18e0819d2cc4a548af63bfb501bbab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e18e0819d2cc4a548af63bfb501bbab8" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 } health_report { overall_health: UNKNOWN } } }
I20260806 18:35:11.128604 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:11.124941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:43141 (local address 127.18.125.129:47849)
0806 18:35:11.125324 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:11.125336 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:11.125494 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:11.125766 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:11.125768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:11.125776 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:35:11.125889 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:11.125895 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:11.126651 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:11.126654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:11.127293 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:11.127301 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:11.127780 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:11.127783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:11.127891 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:11.127937 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:35:11.128302 (+ 365us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:35:11.128311 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 18:35:11.128371 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 18:35:11.128476 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":134,"threads_started":1}
I20260806 18:35:11.128599 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:11.125095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:47849 (local address 127.18.125.130:43141)
0806 18:35:11.125256 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:11.125261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:11.125406 (+ 145us) server_negotiation.cc:407] Connection header received
0806 18:35:11.125607 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:11.125610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:11.125668 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:11.125752 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:11.126005 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:11.126532 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:11.127416 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:11.127657 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:11.127776 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:11.128098 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:11.128208 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:11.128295 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:11.128391 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 18:35:11.128449 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:35:11.134872 27393 raft_consensus.cc:1261] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Refusing update from remote peer e18e0819d2cc4a548af63bfb501bbab8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:11.134981 27238 raft_consensus.cc:1261] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Refusing update from remote peer e18e0819d2cc4a548af63bfb501bbab8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:11.135321 27563 consensus_queue.cc:1035] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c528a6718464a6e80337257e704ce4e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:11.135519 27558 consensus_queue.cc:1035] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1ff8761a16d4560b2688ff332e3b6b6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 43141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:11.154429 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:11.151093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:47597 (local address 127.18.125.129:44399)
0806 18:35:11.151214 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:11.151220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:11.151281 (+ 61us) server_negotiation.cc:407] Connection header received
0806 18:35:11.151445 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:11.151448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:11.151511 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:11.151599 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:11.151835 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:11.152538 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:11.153365 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:11.153584 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:11.153664 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:11.153911 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:11.154001 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:11.154087 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:11.154218 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 18:35:11.154294 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:35:11.154495 27537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:11.151049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44399 (local address 127.18.125.130:47597)
0806 18:35:11.151192 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:11.151225 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:11.151323 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:11.151627 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:11.151630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:11.151639 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:35:11.151757 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:11.151762 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:11.152645 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:11.152648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:11.153253 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:11.153261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:11.153688 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:11.153690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:11.153755 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:11.153784 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:35:11.154110 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:35:11.154119 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 18:35:11.154186 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:35:11.154306 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 18:35:11.176559 26908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:11.176723 26908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:11.178869 26908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e06741026684d59a5b3683f3b62e760: Sending DeleteTablet for 3 replicas of tablet 105a267d54d046e3a31b9ebb0196a236
I20260806 18:35:11.179526 27216 tablet_service.cc:1514] Processing DeleteTablet for tablet 105a267d54d046e3a31b9ebb0196a236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:11 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:45150
I20260806 18:35:11.179526 27059 tablet_service.cc:1514] Processing DeleteTablet for tablet 105a267d54d046e3a31b9ebb0196a236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:11 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:50650
I20260806 18:35:11.179716 27501 tablet_replica.cc:330] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6: stopping tablet replica
I20260806 18:35:11.179716 27500 tablet_replica.cc:330] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8: stopping tablet replica
I20260806 18:35:11.179675 27369 tablet_service.cc:1514] Processing DeleteTablet for tablet 105a267d54d046e3a31b9ebb0196a236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:11 UTC) from {username='oryx', principal='oryx/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:48502
I20260806 18:35:11.179826 27500 raft_consensus.cc:2229] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:35:11.179853 27501 raft_consensus.cc:2229] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:11.179935 27502 tablet_replica.cc:330] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e: stopping tablet replica
I20260806 18:35:11.179968 27501 raft_consensus.cc:2258] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:11.180054 27500 raft_consensus.cc:2258] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:11.180081 27502 raft_consensus.cc:2229] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:11.180250 27502 raft_consensus.cc:2258] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:11.180794 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 26985
I20260806 18:35:11.180794 27501 ts_tablet_manager.cc:1905] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:11.180981 27500 ts_tablet_manager.cc:1905] T 105a267d54d046e3a31b9ebb0196a236 P e18e0819d2cc4a548af63bfb501bbab8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:11.181329 27502 ts_tablet_manager.cc:1905] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:11.183717 27501 ts_tablet_manager.cc:1918] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:35:11.183806 27501 log.cc:1196] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/105a267d54d046e3a31b9ebb0196a236
I20260806 18:35:11.184352 27502 ts_tablet_manager.cc:1918] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:35:11.184463 27502 log.cc:1196] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/105a267d54d046e3a31b9ebb0196a236
I20260806 18:35:11.184803 27502 ts_tablet_manager.cc:1939] T 105a267d54d046e3a31b9ebb0196a236 P 9c528a6718464a6e80337257e704ce4e: Deleting consensus metadata
I20260806 18:35:11.186182 26898 catalog_manager.cc:4841] TS 9c528a6718464a6e80337257e704ce4e (127.18.125.131:44057): tablet 105a267d54d046e3a31b9ebb0196a236 (table test-table [id=b109035a3c0f493b964e30afc1bbb295]) successfully deleted
I20260806 18:35:11.186692 27501 ts_tablet_manager.cc:1939] T 105a267d54d046e3a31b9ebb0196a236 P f1ff8761a16d4560b2688ff332e3b6b6: Deleting consensus metadata
I20260806 18:35:11.187506 26898 catalog_manager.cc:4841] TS f1ff8761a16d4560b2688ff332e3b6b6 (127.18.125.130:43141): tablet 105a267d54d046e3a31b9ebb0196a236 (table test-table [id=b109035a3c0f493b964e30afc1bbb295]) successfully deleted
W20260806 18:35:11.194458 26897 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:44399 (error 108)
I20260806 18:35:11.194731 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 27147
I20260806 18:35:11.197929 27577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:11.196932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44399 (local address 127.0.0.1:50672)
0806 18:35:11.197690 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:11.197786 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:44399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":86,"threads_started":1}
W20260806 18:35:11.198256 26897 catalog_manager.cc:4584] TS e18e0819d2cc4a548af63bfb501bbab8 (127.18.125.129:44399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 105a267d54d046e3a31b9ebb0196a236: Network error: Client connection negotiation failed: client connection to 127.18.125.129:44399: connect: Connection refused (error 111)
I20260806 18:35:11.205109 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 27301
I20260806 18:35:11.213716 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 26878
2026-08-06T18:35:11Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3633 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:35:11 dist-test-slave-8w9v krb5kdc[27580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:35:11 dist-test-slave-8w9v krb5kdc[27580](info): set up 2 sockets
Aug 06 18:35:11 dist-test-slave-8w9v krb5kdc[27580](info): commencing operation
krb5kdc: starting...
W20260806 18:35:12.330422 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:35:12 dist-test-slave-8w9v krb5kdc[27580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:35:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:35:12.463948 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:40997
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:40997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:12.578096 27594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:12.578447 27594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:12.578524 27594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:12.582850 27594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:12.582943 27594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:12.582984 27594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:12.583084 27594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:12.583133 27594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:12.589004 27594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:40997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:40997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:12.590137 27594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:12.591207 27594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:12.599900 27601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:12.599896 27600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:12.599907 27603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:12.600337 27594 server_base.cc:1034] running on GCE node
I20260806 18:35:12.600978 27594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:12.602197 27594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:12.603401 27594 hybrid_clock.cc:648] HybridClock initialized: now 1786041312603385 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T18:35:12Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1396 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:35:12 dist-test-slave-8w9v krb5kdc[27608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:35:12 dist-test-slave-8w9v krb5kdc[27608](info): set up 2 sockets
Aug 06 18:35:12 dist-test-slave-8w9v krb5kdc[27608](info): commencing operation
krb5kdc: starting...
W20260806 18:35:13.689843 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:35:13 dist-test-slave-8w9v krb5kdc[27608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:35:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:35:13.837047 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:44671
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:44671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:13.955706 27621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:13.956027 27621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:13.956072 27621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:13.960259 27621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:13.960338 27621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:13.960358 27621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:13.960376 27621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:13.960392 27621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:13.966104 27621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:44671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:44671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:13.967259 27621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:13.968313 27621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:13.977730 27629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:13.977797 27626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:13.977921 27621 server_base.cc:1034] running on GCE node
W20260806 18:35:13.978298 27627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:13.979477 27621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:13.981263 27621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:13.982474 27621 hybrid_clock.cc:648] HybridClock initialized: now 1786041313982448 us; error 55 us; skew 500 ppm
Aug 06 18:35:13 dist-test-slave-8w9v krb5kdc[27608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041313, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:13.985647 27621 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:35:13.986717 27621 webserver.cc:460] Webserver started at http://127.18.125.190:37911/ using document root <none> and password file <none>
I20260806 18:35:13.987355 27621 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:13.987454 27621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:13.987710 27621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:13.990089 27621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "f1cc44446e764caeb9e051e4e1ad1914"
format_stamp: "Formatted at 2026-08-06 18:35:13 on dist-test-slave-8w9v"
server_key: "0f5aadac9e28432a57f4867ef959ff13"
server_key_iv: "3501d881f34e2f4775bd53dde5bc527e"
server_key_version: "encryptionkey@0"
I20260806 18:35:13.990785 27621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "f1cc44446e764caeb9e051e4e1ad1914"
format_stamp: "Formatted at 2026-08-06 18:35:13 on dist-test-slave-8w9v"
server_key: "0f5aadac9e28432a57f4867ef959ff13"
server_key_iv: "3501d881f34e2f4775bd53dde5bc527e"
server_key_version: "encryptionkey@0"
I20260806 18:35:13.995159 27621 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:35:13.998049 27637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:13.999447 27621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:13.999609 27621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "f1cc44446e764caeb9e051e4e1ad1914"
format_stamp: "Formatted at 2026-08-06 18:35:13 on dist-test-slave-8w9v"
server_key: "0f5aadac9e28432a57f4867ef959ff13"
server_key_iv: "3501d881f34e2f4775bd53dde5bc527e"
server_key_version: "encryptionkey@0"
I20260806 18:35:13.999742 27621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:14.012537 27621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:14.016268 27621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:14.016498 27621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:14.024863 27621 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:44671
I20260806 18:35:14.024919 27691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:44671 every 8 connection(s)
I20260806 18:35:14.025960 27621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:35:14.029112 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:14.035161 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 27621
I20260806 18:35:14.035346 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:35:14.035279 27692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914: Bootstrap starting.
I20260806 18:35:14.035670 18934 external_mini_cluster.cc:1428] Setting key 25708786b40269007ddeac54d373d539
I20260806 18:35:14.038275 27692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:14.039160 27692 log.cc:824] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:14.041440 27692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914: No bootstrap required, opened a new log
Aug 06 18:35:14 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:35:14.046551 27692 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:35:14.046888 27692 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:14.046955 27692 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cc44446e764caeb9e051e4e1ad1914, State: Initialized, Role: FOLLOWER
I20260806 18:35:14.047587 27692 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:35:14.047766 27692 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:35:14.047832 27692 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:35:14.047940 27692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:14.049324 27692 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:35:14.049759 27692 leader_election.cc:302] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1cc44446e764caeb9e051e4e1ad1914; no voters:
I20260806 18:35:14.050165 27692 leader_election.cc:288] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:35:14.050335 27698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:14.050658 27698 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 1 LEADER]: Becoming Leader. State: Replica: f1cc44446e764caeb9e051e4e1ad1914, State: Running, Role: LEADER
I20260806 18:35:14.051088 27698 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:35:14.051689 27692 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:14.052484 27695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:14.037212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.18.125.190:44671)
0806 18:35:14.037681 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:14.037699 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:14.037738 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:35:14.038496 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:14.038530 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:14.038889 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:14.039192 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:14.040217 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.041223 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.042425 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.042919 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.043063 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:14.046124 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:14.046131 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:14.046150 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:14.046188 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:14.048364 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:14.048927 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:14.048937 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:14.048946 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:14.049107 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:14.049405 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:14.049408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:14.049410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:14.050011 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:14.050241 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:14.051093 (+ 852us) server_negotiation.cc:299] Negotiation successful
0806 18:35:14.051433 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":167,"threads_started":1}
I20260806 18:35:14.052492 27698 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1cc44446e764caeb9e051e4e1ad1914. Latest consensus state: current_term: 1 leader_uuid: "f1cc44446e764caeb9e051e4e1ad1914" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } } }
I20260806 18:35:14.052716 27698 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:14.052963 27699 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1cc44446e764caeb9e051e4e1ad1914" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } } }
I20260806 18:35:14.053072 27699 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:14.053140 27710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:14.056615 27710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:14.062383 27710 catalog_manager.cc:1347] Generated new cluster ID: c6000a46230e45378858987ba1073b65
I20260806 18:35:14.062474 27710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:14.081504 27710 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:35:14.082623 27710 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:14.090001 27710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914: Generated new TSK 0
I20260806 18:35:14.090953 27710 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:35:14.133009 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:14.256609 27723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:14.256930 27723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:14.256985 27723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:14.261391 27723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:14.261472 27723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:14.261538 27723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:35:14.267132 27723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:14.268370 27723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:14.269459 27723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:14.280213 27730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:14.280362 27729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:14.280264 27732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:14.280748 27723 server_base.cc:1034] running on GCE node
I20260806 18:35:14.281546 27723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:14.282572 27723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:14.283828 27723 hybrid_clock.cc:648] HybridClock initialized: now 1786041314283782 us; error 63 us; skew 500 ppm
Aug 06 18:35:14 dist-test-slave-8w9v krb5kdc[27608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041314, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:14.287281 27723 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:35:14.288388 27723 webserver.cc:460] Webserver started at http://127.18.125.129:43793/ using document root <none> and password file <none>
I20260806 18:35:14.289021 27723 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:14.289124 27723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:14.289384 27723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:14.291662 27723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "d94709780bd448a3877c2bbe0ad963be"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "06522830e1897c5daaadff2c05da4f57"
server_key_iv: "5cda09b11352b08b09cbab94e14e1219"
server_key_version: "encryptionkey@0"
I20260806 18:35:14.292383 27723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "d94709780bd448a3877c2bbe0ad963be"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "06522830e1897c5daaadff2c05da4f57"
server_key_iv: "5cda09b11352b08b09cbab94e14e1219"
server_key_version: "encryptionkey@0"
I20260806 18:35:14.296710 27723 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:35:14.299515 27739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:14.300941 27723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:14.301090 27723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "d94709780bd448a3877c2bbe0ad963be"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "06522830e1897c5daaadff2c05da4f57"
server_key_iv: "5cda09b11352b08b09cbab94e14e1219"
server_key_version: "encryptionkey@0"
I20260806 18:35:14.301223 27723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:14.314469 27723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:14.317786 27723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:14.318033 27723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:14.318689 27723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:14.319588 27723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:14.319664 27723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:14.319741 27723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:14.319793 27723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:14.329290 27723 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:34851
I20260806 18:35:14.329330 27854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:34851 every 8 connection(s)
I20260806 18:35:14.330222 27723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:14.331058 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 27723
I20260806 18:35:14.331192 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:35:14.331545 18934 external_mini_cluster.cc:1428] Setting key 2c78021acba356778087d5062ff0657d
Aug 06 18:35:14 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041314, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
I20260806 18:35:14.347018 27695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:14.332300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:40397 (local address 127.18.125.190:44671)
0806 18:35:14.332466 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:14.332471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:14.333447 (+ 976us) server_negotiation.cc:407] Connection header received
0806 18:35:14.335443 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:14.335449 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:14.335516 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:14.335620 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:14.336918 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.337512 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.339427 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.339660 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.339747 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:14.342932 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:14.342936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:14.342938 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:14.342973 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:14.344399 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:14.345008 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:14.345011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:14.345012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:14.345093 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:14.345544 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:14.345546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:14.345547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:14.345738 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:14.345817 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:14.346648 (+ 831us) server_negotiation.cc:299] Negotiation successful
0806 18:35:14.346800 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 18:35:14.347862 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:14.332712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:40397)
0806 18:35:14.333273 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:14.333333 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:14.335176 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:14.335778 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:14.335784 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:14.336235 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:14.336745 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:14.336758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.337665 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.337672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:14.339274 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:14.339290 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.339845 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.339851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:14.340109 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:14.340846 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:14.340873 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:14.342734 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:14.344533 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:14.344539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:14.344553 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:14.344869 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:14.345188 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:14.345190 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:14.345192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:14.345432 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:14.345876 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:14.345883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:14.346322 (+ 439us) client_negotiation.cc:769] Sending connection context
0806 18:35:14.346557 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 18:35:14.346882 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":174,"threads_started":1}
I20260806 18:35:14.349440 27855 heartbeater.cc:347] Connected to a master server at 127.18.125.190:44671
I20260806 18:35:14.349756 27855 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:14.350410 27855 heartbeater.cc:511] Master 127.18.125.190:44671 requested a full tablet report, sending...
I20260806 18:35:14.352346 27652 ts_manager.cc:194] Registered new tserver with Master: d94709780bd448a3877c2bbe0ad963be (127.18.125.129:34851)
I20260806 18:35:14.354130 27652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:40397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:35:14.367952 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:14.489477 27860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:14.489826 27860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:14.489920 27860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:14.494534 27860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:14.494649 27860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:14.494762 27860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:35:14.500692 27860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:14.501919 27860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:14.502998 27860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:14.514487 27865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:14.515038 27860 server_base.cc:1034] running on GCE node
W20260806 18:35:14.515167 27866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:14.515347 27868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:14.515843 27860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:14.516739 27860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:14.517961 27860 hybrid_clock.cc:648] HybridClock initialized: now 1786041314517922 us; error 52 us; skew 500 ppm
Aug 06 18:35:14 dist-test-slave-8w9v krb5kdc[27608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041314, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:14.521186 27860 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:35:14.522328 27860 webserver.cc:460] Webserver started at http://127.18.125.130:42321/ using document root <none> and password file <none>
I20260806 18:35:14.522989 27860 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:14.523095 27860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:14.523360 27860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:14.525715 27860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "0db20541311946ddba0948bf92851123"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "565ec8c8ca2534d83ceb1e90b5cb40da"
server_key_iv: "98dd06545b4f82bd29c6878fe6523768"
server_key_version: "encryptionkey@0"
I20260806 18:35:14.526340 27860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "0db20541311946ddba0948bf92851123"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "565ec8c8ca2534d83ceb1e90b5cb40da"
server_key_iv: "98dd06545b4f82bd29c6878fe6523768"
server_key_version: "encryptionkey@0"
I20260806 18:35:14.530540 27860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:35:14.533254 27875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:14.534571 27860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:14.534746 27860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "0db20541311946ddba0948bf92851123"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "565ec8c8ca2534d83ceb1e90b5cb40da"
server_key_iv: "98dd06545b4f82bd29c6878fe6523768"
server_key_version: "encryptionkey@0"
I20260806 18:35:14.534878 27860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:14.546308 27860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:14.549808 27860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:14.550032 27860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:14.550678 27860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:14.551636 27860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:14.551707 27860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:14.551784 27860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:14.551837 27860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:14.562216 27860 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:41443
I20260806 18:35:14.562252 27998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:41443 every 8 connection(s)
I20260806 18:35:14.563174 27860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:14.565872 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 27860
I20260806 18:35:14.566006 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:35:14.566321 18934 external_mini_cluster.cc:1428] Setting key 7c74e2e2e00f1ef216c134ba9fe16af0
Aug 06 18:35:14 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041314, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:35:14.580641 27695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:14.565456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:38913 (local address 127.18.125.190:44671)
0806 18:35:14.565645 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:14.565653 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:14.566668 (+ 1015us) server_negotiation.cc:407] Connection header received
0806 18:35:14.568339 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:14.568343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:14.568398 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:14.568501 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:14.569781 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.570693 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.571965 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.572434 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.572567 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:14.575578 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:14.575582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:14.575585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:14.575620 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:14.577477 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:14.578727 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:14.578730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:14.578732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:14.578821 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:14.579328 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:14.579331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:14.579332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:14.579548 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:14.579636 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:14.580369 (+ 733us) server_negotiation.cc:299] Negotiation successful
0806 18:35:14.580504 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
I20260806 18:35:14.581756 28001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:14.565874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:38913)
0806 18:35:14.566475 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:14.566534 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:14.568046 (+ 1512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:14.568669 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:14.568677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:14.569094 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:14.569611 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:14.569625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.570856 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.570859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:14.571823 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:14.571833 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.572605 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.572608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:14.572768 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:14.573478 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:14.573497 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:14.575352 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:14.577644 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:14.577650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:14.577666 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:14.578578 (+ 912us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:14.578992 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:14.578998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:14.579003 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:14.579218 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:14.579693 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:14.579699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:14.580006 (+ 307us) client_negotiation.cc:769] Sending connection context
0806 18:35:14.580327 (+ 321us) client_negotiation.cc:240] Negotiation successful
0806 18:35:14.580710 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":207,"threads_started":1}
I20260806 18:35:14.583220 27999 heartbeater.cc:347] Connected to a master server at 127.18.125.190:44671
I20260806 18:35:14.583513 27999 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:14.584067 27999 heartbeater.cc:511] Master 127.18.125.190:44671 requested a full tablet report, sending...
I20260806 18:35:14.585369 27652 ts_manager.cc:194] Registered new tserver with Master: 0db20541311946ddba0948bf92851123 (127.18.125.130:41443)
I20260806 18:35:14.586287 27652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:38913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:35:14.599895 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:14.715312 28004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:14.715597 28004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:14.715714 28004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:14.719899 28004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:14.719981 28004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:14.720044 28004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:35:14.725607 28004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:14.726794 28004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:14.727793 28004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:14.738246 28009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:14.738313 28004 server_base.cc:1034] running on GCE node
W20260806 18:35:14.738263 28013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:14.738260 28010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:14.738951 28004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:14.739681 28004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:14.740882 28004 hybrid_clock.cc:648] HybridClock initialized: now 1786041314740863 us; error 32 us; skew 500 ppm
Aug 06 18:35:14 dist-test-slave-8w9v krb5kdc[27608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041314, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:14.743633 28004 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:35:14.744653 28004 webserver.cc:460] Webserver started at http://127.18.125.131:43005/ using document root <none> and password file <none>
I20260806 18:35:14.745251 28004 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:14.745347 28004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:14.745586 28004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:14.747684 28004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "903393d710494e7ca773863250932c2e"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "d1b1f9d9b61fb2be3bab2c24d64f1daf"
server_key_iv: "375ef0ad35165adf1aa1f2d6531cd701"
server_key_version: "encryptionkey@0"
I20260806 18:35:14.748320 28004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "903393d710494e7ca773863250932c2e"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "d1b1f9d9b61fb2be3bab2c24d64f1daf"
server_key_iv: "375ef0ad35165adf1aa1f2d6531cd701"
server_key_version: "encryptionkey@0"
I20260806 18:35:14.752075 28004 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:14.754869 28020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:14.756103 28004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:35:14.756287 28004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "903393d710494e7ca773863250932c2e"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "d1b1f9d9b61fb2be3bab2c24d64f1daf"
server_key_iv: "375ef0ad35165adf1aa1f2d6531cd701"
server_key_version: "encryptionkey@0"
I20260806 18:35:14.756404 28004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:14.785259 28004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:14.789215 28004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:14.789409 28004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:14.790006 28004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:14.790920 28004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:14.791064 28004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:14.791149 28004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:14.791174 28004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:14.801618 28004 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:40855
I20260806 18:35:14.801697 28137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:40855 every 8 connection(s)
I20260806 18:35:14.802665 28004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:14.807808 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 28004
I20260806 18:35:14.807929 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:35:14.808324 18934 external_mini_cluster.cc:1428] Setting key fb9bd3f39c3598941181060efc653785
Aug 06 18:35:14 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041314, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:35:14.819780 27695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:14.804813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:48281 (local address 127.18.125.190:44671)
0806 18:35:14.805009 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:14.805014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:14.805962 (+ 948us) server_negotiation.cc:407] Connection header received
0806 18:35:14.807325 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:14.807327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:14.807401 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:14.807481 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:14.808961 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.809728 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.811209 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.811461 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.811559 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:14.814492 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:14.814496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:14.814498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:14.814534 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:14.816351 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:14.817209 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:14.817221 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:14.817223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:14.817323 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:14.817745 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:14.817748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:14.817750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:14.818161 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:14.818406 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:14.819250 (+ 844us) server_negotiation.cc:299] Negotiation successful
0806 18:35:14.819461 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 18:35:14.821410 28141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:14.805272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:48281)
0806 18:35:14.805809 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:14.805852 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:14.807117 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:14.807617 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:14.807625 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:14.808100 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:14.808800 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:14.808827 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.809871 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.809875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:14.811066 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:14.811076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:14.811630 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:14.811633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:14.811794 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:14.812558 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:14.812576 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:14.814281 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:14.816518 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:14.816526 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:14.816546 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:14.816865 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:14.817450 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:14.817454 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:14.817456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:14.817624 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:14.818414 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:14.818429 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:14.818898 (+ 469us) client_negotiation.cc:769] Sending connection context
0806 18:35:14.819190 (+ 292us) client_negotiation.cc:240] Negotiation successful
0806 18:35:14.819851 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":188,"threads_started":1}
I20260806 18:35:14.823189 28138 heartbeater.cc:347] Connected to a master server at 127.18.125.190:44671
I20260806 18:35:14.823503 28138 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:14.824394 28138 heartbeater.cc:511] Master 127.18.125.190:44671 requested a full tablet report, sending...
I20260806 18:35:14.825699 27652 ts_manager.cc:194] Registered new tserver with Master: 903393d710494e7ca773863250932c2e (127.18.125.131:40855)
I20260806 18:35:14.826341 27652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:48281
I20260806 18:35:14.835551 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab.
Entry for principal oryx/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab.
WARNING: no policy specified for oryx/127.18.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab.
Entry for principal oryx/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab.
WARNING: no policy specified for oryx/127.18.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab.
Entry for principal oryx/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab.
I20260806 18:35:14.919577 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 27723
I20260806 18:35:14.926982 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 27860
I20260806 18:35:14.933995 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 28004
I20260806 18:35:14.942812 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 27621
I20260806 18:35:14.952800 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:44671
--webserver_interface=127.18.125.190
--webserver_port=37911
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:44671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:15.070269 28149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:15.070626 28149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:15.070705 28149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:15.074836 28149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:15.074951 28149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:15.074991 28149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:15.075031 28149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:15.075067 28149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:15.080721 28149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:44671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:44671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=37911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:15.081914 28149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:15.083011 28149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:35:15.091907 28149 server_base.cc:1034] running on GCE node
W20260806 18:35:15.091890 28157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:15.091895 28155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:15.092092 28154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:15.092800 28149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:15.094260 28149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:15.095494 28149 hybrid_clock.cc:648] HybridClock initialized: now 1786041315095389 us; error 121 us; skew 500 ppm
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041315, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:15.098627 28149 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:35:15.099947 28149 webserver.cc:460] Webserver started at http://127.18.125.190:37911/ using document root <none> and password file <none>
I20260806 18:35:15.100714 28149 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:15.100801 28149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:15.105130 28149 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:35:15.107613 28164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:15.109050 28149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:15.109217 28149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "f1cc44446e764caeb9e051e4e1ad1914"
format_stamp: "Formatted at 2026-08-06 18:35:13 on dist-test-slave-8w9v"
server_key: "0f5aadac9e28432a57f4867ef959ff13"
server_key_iv: "3501d881f34e2f4775bd53dde5bc527e"
server_key_version: "encryptionkey@0"
I20260806 18:35:15.109822 28149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:15.119623 28149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:15.122979 28149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:15.123199 28149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:15.130897 28149 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:44671
I20260806 18:35:15.130926 28222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:44671 every 8 connection(s)
I20260806 18:35:15.132081 28149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:35:15.136758 28223 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:35:15.138487 28223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914: Bootstrap starting.
I20260806 18:35:15.140000 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 28149
I20260806 18:35:15.140534 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:34851
--local_ip_for_outbound_sockets=127.18.125.129
--tserver_master_addrs=127.18.125.190:44671
--webserver_port=43793
--webserver_interface=127.18.125.129
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.125.129
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:35:15.156726 28223 log.cc:824] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:15.163666 28223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:35:15.164080 28223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914: Bootstrap complete.
I20260806 18:35:15.168459 28223 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:35:15.168877 28223 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1cc44446e764caeb9e051e4e1ad1914, State: Initialized, Role: FOLLOWER
I20260806 18:35:15.169385 28223 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:35:15.169498 28223 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:35:15.169584 28223 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:35:15.169680 28223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:35:15.171653 28223 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:35:15.171995 28223 leader_election.cc:302] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1cc44446e764caeb9e051e4e1ad1914; no voters:
I20260806 18:35:15.172376 28223 leader_election.cc:288] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:35:15.172465 28228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:35:15.172641 28228 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [term 2 LEADER]: Becoming Leader. State: Replica: f1cc44446e764caeb9e051e4e1ad1914, State: Running, Role: LEADER
I20260806 18:35:15.172945 28228 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:35:15.173657 28223 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:15.174294 28229 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f1cc44446e764caeb9e051e4e1ad1914" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } } }
I20260806 18:35:15.174453 28229 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:15.175083 28230 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1cc44446e764caeb9e051e4e1ad1914. Latest consensus state: current_term: 2 leader_uuid: "f1cc44446e764caeb9e051e4e1ad1914" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1cc44446e764caeb9e051e4e1ad1914" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } } }
I20260806 18:35:15.175184 28230 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914 [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:15.175381 28235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:15.178864 28235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:15.179587 28235 catalog_manager.cc:1259] Loaded cluster ID: c6000a46230e45378858987ba1073b65
I20260806 18:35:15.179649 28235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:15.182055 28235 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:15.182745 28235 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f1cc44446e764caeb9e051e4e1ad1914: Loaded TSK: 0
I20260806 18:35:15.183954 28235 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:35:15.265900 28226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:15.266191 28226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:15.266239 28226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:15.270622 28226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:15.270699 28226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:15.270762 28226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:35:15.276698 28226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.129.keytab
--principal=oryx/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:34851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=43793
--tserver_master_addrs=127.18.125.190:44671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:15.277841 28226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:15.278893 28226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:15.289799 28252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:15.290058 28251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:15.290210 28226 server_base.cc:1034] running on GCE node
W20260806 18:35:15.289891 28255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:15.290692 28226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:15.291446 28226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:15.292630 28226 hybrid_clock.cc:648] HybridClock initialized: now 1786041315292599 us; error 51 us; skew 500 ppm
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041315, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:15.295668 28226 init.cc:377] Logged in from keytab as oryx/127.18.125.129@KRBTEST.COM (short username oryx)
I20260806 18:35:15.296834 28226 webserver.cc:460] Webserver started at http://127.18.125.129:43793/ using document root <none> and password file <none>
I20260806 18:35:15.297531 28226 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:15.297619 28226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:15.301891 28226 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:35:15.304306 28262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:15.305658 28226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:15.305823 28226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "d94709780bd448a3877c2bbe0ad963be"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "06522830e1897c5daaadff2c05da4f57"
server_key_iv: "5cda09b11352b08b09cbab94e14e1219"
server_key_version: "encryptionkey@0"
I20260806 18:35:15.306452 28226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:15.341037 28226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:15.344642 28226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:15.344872 28226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:15.345532 28226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:15.346428 28226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:15.346477 28226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:15.346546 28226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:15.346578 28226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:15.356019 28226 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:34851
I20260806 18:35:15.356040 28383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:34851 every 8 connection(s)
I20260806 18:35:15.357102 28226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:15.358649 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 28226
I20260806 18:35:15.359196 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:41443
--local_ip_for_outbound_sockets=127.18.125.130
--tserver_master_addrs=127.18.125.190:44671
--webserver_port=42321
--webserver_interface=127.18.125.130
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.125.130
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:35:15.371466 28387 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:15.359407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:50175 (local address 127.18.125.190:44671)
0806 18:35:15.359929 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:15.359956 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:15.360335 (+ 379us) server_negotiation.cc:407] Connection header received
0806 18:35:15.361912 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:15.361928 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:15.362224 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:15.362565 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:15.363766 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.364831 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.366021 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.366502 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.366690 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.370660 (+ 3970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:50175: BlockingRecv error: recv got EOF from 127.18.125.129:50175 (error 108)
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":126,"threads_started":1}
I20260806 18:35:15.371474 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:15.359601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:50175)
0806 18:35:15.360110 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:15.360206 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:15.361474 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:15.362637 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:15.362645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:15.363101 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:15.363573 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.363586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.364989 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.364992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.365880 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.365889 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.366681 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.366686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.366883 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.367587 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:15.367604 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:15.370412 (+ 2808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":147,"threads_started":1}
W20260806 18:35:15.372262 28384 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:44671 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.125.190:44671: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:15.378759 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:15.372770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:56115)
0806 18:35:15.372953 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:15.372968 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:15.373039 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:15.373383 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:15.373388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:15.373595 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:15.373720 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.373727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.374555 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.374558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.375457 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.375469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.375908 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.375911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.376015 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.376520 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:15.376534 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:15.378615 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110}
W20260806 18:35:15.378903 28387 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:15.372857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:56115 (local address 127.18.125.190:44671)
0806 18:35:15.372979 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:15.372983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:15.373013 (+ 30us) server_negotiation.cc:407] Connection header received
0806 18:35:15.373135 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:15.373138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:15.373231 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:15.373314 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:15.373837 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.374453 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.375580 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.375797 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.375879 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.378755 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:56115: BlockingRecv error: recv got EOF from 127.18.125.129:56115 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:15.384991 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:15.379517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:43543)
0806 18:35:15.379680 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:15.379694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:15.379763 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:15.380000 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:15.380003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:15.380200 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:15.380292 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.380298 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.381000 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.381006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.381880 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.381893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.382315 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.382318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.382416 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.382881 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:15.382895 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:15.384859 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260806 18:35:15.385133 28387 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:15.379597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:43543 (local address 127.18.125.190:44671)
0806 18:35:15.379715 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:15.379718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:15.379730 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:35:15.379847 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:15.379849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:15.379888 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:15.379960 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:15.380399 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.380884 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.382003 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.382207 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.382287 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.384993 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:43543: BlockingRecv error: recv got EOF from 127.18.125.129:43543 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 18:35:15.385368 28384 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:35:15.479694 28389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:15.479996 28389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:15.480042 28389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:15.484234 28389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:15.484308 28389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:15.484369 28389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:35:15.490242 28389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.130.keytab
--principal=oryx/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:41443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=42321
--tserver_master_addrs=127.18.125.190:44671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:15.491381 28389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:15.492501 28389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:15.502772 28395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:15.502791 28394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:15.502831 28389 server_base.cc:1034] running on GCE node
W20260806 18:35:15.502861 28397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:15.503458 28389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:15.504170 28389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:15.505370 28389 hybrid_clock.cc:648] HybridClock initialized: now 1786041315505339 us; error 25 us; skew 500 ppm
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041315, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:15.508217 28389 init.cc:377] Logged in from keytab as oryx/127.18.125.130@KRBTEST.COM (short username oryx)
I20260806 18:35:15.509109 28389 webserver.cc:460] Webserver started at http://127.18.125.130:42321/ using document root <none> and password file <none>
I20260806 18:35:15.509721 28389 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:15.509828 28389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:15.514024 28389 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:35:15.516302 28405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:15.517547 28389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:35:15.517690 28389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "0db20541311946ddba0948bf92851123"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "565ec8c8ca2534d83ceb1e90b5cb40da"
server_key_iv: "98dd06545b4f82bd29c6878fe6523768"
server_key_version: "encryptionkey@0"
I20260806 18:35:15.518265 28389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:15.564318 28389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:15.567426 28389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:15.567662 28389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:15.568341 28389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:15.569243 28389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:15.569327 28389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:15.569410 28389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:15.569464 28389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:15.579895 28389 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:41443
I20260806 18:35:15.579916 28521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:41443 every 8 connection(s)
I20260806 18:35:15.581010 28389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:15.587575 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 28389
I20260806 18:35:15.588140 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:40855
--local_ip_for_outbound_sockets=127.18.125.131
--tserver_master_addrs=127.18.125.190:44671
--webserver_port=43005
--webserver_interface=127.18.125.131
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.125.131
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:35:15.593256 28387 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:15.582965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:39477 (local address 127.18.125.190:44671)
0806 18:35:15.583116 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:15.583120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:15.583960 (+ 840us) server_negotiation.cc:407] Connection header received
0806 18:35:15.585293 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:15.585297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:15.585352 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:15.585432 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:15.586684 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.587292 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.589115 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.589349 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.589451 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.593120 (+ 3669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:39477: BlockingRecv error: recv got EOF from 127.18.125.130:39477 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:35:15.593951 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:15.583301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:39477)
0806 18:35:15.583786 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:15.583832 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:15.585084 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:15.585610 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:15.585616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:15.585991 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:15.586495 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.586509 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.587456 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.587463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.588962 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.588979 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.589532 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.589539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.589790 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.590473 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:15.590490 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:15.592973 (+ 2483us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":136,"threads_started":1}
W20260806 18:35:15.594456 28522 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:44671 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.125.190:44671: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:15.600343 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:15.595010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:35823)
0806 18:35:15.595136 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:15.595154 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:15.595246 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:15.595536 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:15.595539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:15.595704 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:15.595800 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.595806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.596651 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.596654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.597188 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.597196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.597653 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.597657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.597749 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.598259 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:15.598271 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:15.600180 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:35:15.600459 28387 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:15.595089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:35823 (local address 127.18.125.190:44671)
0806 18:35:15.595215 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:15.595219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:15.595233 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:35:15.595369 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:15.595372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:15.595429 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:15.595518 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:15.595921 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.596521 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.597309 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.597536 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.597649 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.600328 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:35823: BlockingRecv error: recv got EOF from 127.18.125.130:35823 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:35:15.607132 28387 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:15.601167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:51325 (local address 127.18.125.190:44671)
0806 18:35:15.601306 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:15.601310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:15.601494 (+ 184us) server_negotiation.cc:407] Connection header received
0806 18:35:15.601606 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:15.601608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:15.601649 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:15.601715 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:15.602150 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.602654 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.603749 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.603963 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.604037 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.607016 (+ 2979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:51325: BlockingRecv error: recv got EOF from 127.18.125.130:51325 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 18:35:15.607215 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:15.601085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:51325)
0806 18:35:15.601406 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:15.601422 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:15.601519 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:15.601759 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:15.601762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:15.601943 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:15.602042 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.602048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.602752 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.602756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.603631 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.603641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.604075 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.604079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.604202 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.604678 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:15.604692 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:15.606853 (+ 2161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239}
W20260806 18:35:15.607484 28522 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:35:15.708841 28527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:15.709152 28527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:15.709200 28527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:15.713567 28527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:15.713689 28527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:15.713790 28527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:35:15.719599 28527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/krb5kdc/oryx_127.18.125.131.keytab
--principal=oryx/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:40855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=43005
--tserver_master_addrs=127.18.125.190:44671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:15.720889 28527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:15.722002 28527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:15.732124 28534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:15.732174 28533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:15.732227 28536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:15.732565 28527 server_base.cc:1034] running on GCE node
I20260806 18:35:15.733114 28527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:15.733834 28527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:15.735064 28527 hybrid_clock.cc:648] HybridClock initialized: now 1786041315735045 us; error 30 us; skew 500 ppm
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041315, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:15.738364 28527 init.cc:377] Logged in from keytab as oryx/127.18.125.131@KRBTEST.COM (short username oryx)
I20260806 18:35:15.739392 28527 webserver.cc:460] Webserver started at http://127.18.125.131:43005/ using document root <none> and password file <none>
I20260806 18:35:15.740010 28527 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:15.740128 28527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:15.744444 28527 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 18:35:15.747469 28543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:15.749156 28527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:15.749307 28527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "903393d710494e7ca773863250932c2e"
format_stamp: "Formatted at 2026-08-06 18:35:14 on dist-test-slave-8w9v"
server_key: "d1b1f9d9b61fb2be3bab2c24d64f1daf"
server_key_iv: "375ef0ad35165adf1aa1f2d6531cd701"
server_key_version: "encryptionkey@0"
I20260806 18:35:15.749962 28527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:15.770365 28527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:15.774392 28527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:15.774667 28527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:15.775624 28527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:15.776895 28527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:15.777209 28527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:15.777312 28527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:15.777333 28527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:15.787186 28527 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:40855
I20260806 18:35:15.787205 28661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:40855 every 8 connection(s)
I20260806 18:35:15.788326 28527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:15.796326 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 28527
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:35:15.805666 28387 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:15.791002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:33519 (local address 127.18.125.190:44671)
0806 18:35:15.791265 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:15.791272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:15.792773 (+ 1501us) server_negotiation.cc:407] Connection header received
0806 18:35:15.794199 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:15.794202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:15.794261 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:15.794354 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:15.796372 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.797631 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.799620 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.800179 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.800311 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.805500 (+ 5189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:33519: BlockingRecv error: recv got EOF from 127.18.125.131:33519 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 18:35:15.806233 28664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:15.791637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:33519)
0806 18:35:15.792498 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:15.792568 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:15.793940 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:15.794562 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:15.794570 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:15.795299 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:15.795868 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.795884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.797889 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.797895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.799474 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.799487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.800406 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.800410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.800709 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.801662 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:15.801687 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:15.805053 (+ 3366us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":204,"threads_started":1}
W20260806 18:35:15.807066 28662 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:44671 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.125.190:44671: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
I20260806 18:35:15.809705 28666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:15.797941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55496 (local address 127.18.125.190:44671)
0806 18:35:15.798227 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:15.798230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:15.798244 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:35:15.798333 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:15.798335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:15.798376 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:15.798454 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:15.799436 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.800426 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.801922 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.802183 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.802266 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.804896 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:15.804904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:15.804927 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:15.804970 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:15.807251 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:15.807904 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:15.807909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:15.807915 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:15.808033 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:15.808400 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:15.808405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:15.808407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:15.808779 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:15.809003 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:15.809319 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 18:35:15.809580 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":205,"thread_start_us":96,"threads_started":1}
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:15.813737 28664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:15.807627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:33767)
0806 18:35:15.807837 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:15.807852 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:15.807939 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:15.808202 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:15.808205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:15.808383 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:15.808480 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.808485 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.809240 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.809245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.810172 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.810183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.810708 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.810713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.810816 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.811347 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:15.811360 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:15.813633 (+ 2273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125}
W20260806 18:35:15.814126 28387 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:15.807715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:33767 (local address 127.18.125.190:44671)
0806 18:35:15.807841 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:15.807852 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:15.807900 (+ 48us) server_negotiation.cc:407] Connection header received
0806 18:35:15.808017 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:15.808020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:15.808072 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:15.808187 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:15.808600 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.809126 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.810295 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.810535 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.810650 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.814017 (+ 3367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:33767: BlockingRecv error: recv got EOF from 127.18.125.131:33767 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:15 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:15.820680 28664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:15.814353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:53093)
0806 18:35:15.814485 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:15.814498 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:15.814602 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:15.814989 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:15.814991 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:15.815126 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:15.815214 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.815219 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.815912 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.815915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.816541 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:15.816550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.817350 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.817353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:15.817427 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.817993 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:15.818010 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:15.820537 (+ 2527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 18:35:15.820791 28387 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:15.814596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:53093 (local address 127.18.125.190:44671)
0806 18:35:15.814776 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:15.814779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:15.814805 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:35:15.814843 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:15.814845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:15.814881 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:15.815199 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:15.815300 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.815808 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.816782 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:15.817140 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:15.817302 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:15.820661 (+ 3359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:53093: BlockingRecv error: recv got EOF from 127.18.125.131:53093 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
W20260806 18:35:15.821022 28662 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 18:35:16 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:16 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:16.392026 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:16.386089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:49249)
0806 18:35:16.386405 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:16.386419 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:16.386516 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:16.386804 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:16.386806 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:16.386993 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:16.387094 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:16.387099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.387872 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.387874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:16.388547 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:16.388556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.389227 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.389232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:16.389308 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:16.389869 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:16.389882 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:16.391889 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":115,"threads_started":1}
W20260806 18:35:16.392314 28668 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:16.386167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:49249 (local address 127.18.125.190:44671)
0806 18:35:16.386487 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:16.386491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:16.386508 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:35:16.386614 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:16.386616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:16.386664 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:16.386769 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:16.387207 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.387768 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.388678 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.388922 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.389218 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:16.392124 (+ 2906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:49249: BlockingRecv error: recv got EOF from 127.18.125.129:49249 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":133,"threads_started":1}
Aug 06 18:35:16 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:16 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:16.614944 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:16.608369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:50359)
0806 18:35:16.608822 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:16.608837 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:16.608975 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:16.609328 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:16.609331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:16.609565 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:16.609737 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:16.609767 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.610598 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.610603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:16.611298 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:16.611309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.611849 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.611853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:16.611961 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:16.612542 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:16.612554 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:16.614750 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":113,"threads_started":1}
W20260806 18:35:16.615222 28668 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:16.608542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:50359 (local address 127.18.125.190:44671)
0806 18:35:16.608772 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:16.608777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:16.608988 (+ 211us) server_negotiation.cc:407] Connection header received
0806 18:35:16.609058 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:16.609063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:16.609141 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:16.609253 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:16.609924 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.610475 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.611427 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.611650 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.611842 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:16.614967 (+ 3125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:50359: BlockingRecv error: recv got EOF from 127.18.125.130:50359 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 18:35:16 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:16 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:16.828032 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:16.821773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:54055)
0806 18:35:16.822147 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:16.822162 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:16.822320 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:16.822651 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:16.822655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:16.822880 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:16.822984 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:16.822991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.823785 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.823791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:16.824428 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:16.824437 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.824931 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.824934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:16.825025 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:16.825649 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:16.825663 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:16.827875 (+ 2212us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":96,"threads_started":1}
W20260806 18:35:16.828091 28668 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:16.821915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:54055 (local address 127.18.125.190:44671)
0806 18:35:16.822098 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:16.822103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:16.822261 (+ 158us) server_negotiation.cc:407] Connection header received
0806 18:35:16.822444 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:16.822448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:16.822506 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:16.822600 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:16.823118 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.823666 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.824564 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:16.824801 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:16.824926 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:16.827996 (+ 3070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:54055: BlockingRecv error: recv got EOF from 127.18.125.131:54055 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 18:35:17 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:17 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:17.398640 28671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:17.392933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:46481)
0806 18:35:17.393262 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:17.393278 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:17.393419 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:17.393744 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:17.393747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:17.393948 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:17.394051 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:17.394057 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.394844 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.394847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:17.395429 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:17.395437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.395889 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.395892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:17.395969 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:17.396503 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:17.396515 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:17.398487 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":138,"threads_started":1}
W20260806 18:35:17.399259 28672 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:17.392988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:46481 (local address 127.18.125.190:44671)
0806 18:35:17.393264 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:17.393271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:17.393352 (+ 81us) server_negotiation.cc:407] Connection header received
0806 18:35:17.393528 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:17.393531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:17.393590 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:17.393671 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:17.394170 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.394738 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.395544 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.395779 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.395861 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:17.398958 (+ 3097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:46481: BlockingRecv error: recv got EOF from 127.18.125.129:46481 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":97,"threads_started":1}
Aug 06 18:35:17 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:17 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:17.626916 28673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:17.616100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:44903)
0806 18:35:17.616501 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:17.616517 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:17.616635 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:17.616881 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:17.616884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:17.617085 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:17.617189 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:17.617194 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.617990 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.617992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:17.618572 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:17.618580 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.619039 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.619041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:17.619107 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:17.624523 (+ 5416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:17.624539 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:17.626756 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":139,"threads_started":1}
W20260806 18:35:17.627096 28672 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:17.616165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:44903 (local address 127.18.125.190:44671)
0806 18:35:17.616381 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:17.616387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:17.616609 (+ 222us) server_negotiation.cc:407] Connection header received
0806 18:35:17.616712 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:17.616714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:17.616765 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:17.616851 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:17.617309 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.617886 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.618691 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.618918 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.619021 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:17.626947 (+ 7926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:44903: BlockingRecv error: recv got EOF from 127.18.125.130:44903 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 18:35:17 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:17 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:17.836855 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:17.829222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:45119)
0806 18:35:17.829608 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:17.829628 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:17.829777 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:17.830156 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:17.830160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:17.830379 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:17.830545 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:17.830555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.831899 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.831908 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:17.832793 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:17.832811 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.833480 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.833484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:17.833585 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:17.834283 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:17.834297 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:17.836679 (+ 2382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":160,"threads_started":1}
W20260806 18:35:17.836984 28672 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:17.829534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:45119 (local address 127.18.125.190:44671)
0806 18:35:17.829731 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:17.829738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:17.829771 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:35:17.829946 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:17.829948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:17.830006 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:17.830095 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:17.830728 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.831441 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.833023 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:17.833286 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:17.833405 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:17.836875 (+ 3470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:45119: BlockingRecv error: recv got EOF from 127.18.125.131:45119 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 18:35:18 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:18 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:18.405545 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:18.399517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:41253)
0806 18:35:18.399861 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:18.399876 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:18.399990 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:18.400355 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:18.400357 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:18.400543 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:18.400647 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:18.400652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.401538 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.401541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:18.402133 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:18.402143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.402616 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.402619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:18.402691 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:18.403379 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:18.403392 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:18.405412 (+ 2020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":123,"threads_started":1}
W20260806 18:35:18.405718 28676 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:18.399591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:41253 (local address 127.18.125.190:44671)
0806 18:35:18.399884 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:18.399888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:18.399960 (+ 72us) server_negotiation.cc:407] Connection header received
0806 18:35:18.400113 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:18.400150 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:18.400201 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:18.400294 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:18.400795 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.401370 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.402254 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.402499 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.402580 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:18.405568 (+ 2988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:41253: BlockingRecv error: recv got EOF from 127.18.125.129:41253 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":99,"threads_started":1}
Aug 06 18:35:18 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:18 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:18.634042 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:18.628100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:40301)
0806 18:35:18.628455 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:18.628468 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:18.628619 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:18.628899 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:18.628902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:18.629117 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:18.629214 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:18.629220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.630226 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.630231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:18.630792 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:18.630800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.631273 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.631277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:18.631360 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:18.631953 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:18.631965 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:18.633913 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":89,"threads_started":1}
W20260806 18:35:18.634229 28676 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:18.628238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:40301 (local address 127.18.125.190:44671)
0806 18:35:18.628424 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:18.628429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:18.628556 (+ 127us) server_negotiation.cc:407] Connection header received
0806 18:35:18.628724 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:18.628727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:18.628782 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:18.628873 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:18.629574 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.630110 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.630924 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.631154 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.631273 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:18.634063 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:40301: BlockingRecv error: recv got EOF from 127.18.125.130:40301 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 18:35:18 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:18 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:18.843555 28678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:18.837922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:43845)
0806 18:35:18.838227 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:18.838241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:18.838357 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:18.838601 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:18.838602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:18.838782 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:18.838880 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:18.838885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.839760 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.839762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:18.840457 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:18.840468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.840963 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.840966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:18.841027 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:18.841552 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:18.841565 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:18.843429 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":120,"threads_started":1}
W20260806 18:35:18.843781 28676 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:18.837975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:43845 (local address 127.18.125.190:44671)
0806 18:35:18.838121 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:18.838126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:18.838327 (+ 201us) server_negotiation.cc:407] Connection header received
0806 18:35:18.838442 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:18.838444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:18.838492 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:18.838561 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:18.839043 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.839640 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.840609 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:18.840843 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:18.840945 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:18.843636 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:43845: BlockingRecv error: recv got EOF from 127.18.125.131:43845 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 18:35:19 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:19 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:19.412673 28679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:19.406552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:33471)
0806 18:35:19.406872 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:19.406888 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:19.406993 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:19.407313 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:19.407316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:19.407547 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:19.407648 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:19.407654 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.408632 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.408637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:19.409199 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:19.409207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.409690 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.409694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:19.409815 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:19.410419 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:19.410431 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:19.412541 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":92,"threads_started":1}
W20260806 18:35:19.412905 28680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:19.406700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:33471 (local address 127.18.125.190:44671)
0806 18:35:19.406999 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:19.407005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:19.407049 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:35:19.407097 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:19.407101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:19.407156 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:19.407280 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:19.407785 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.408511 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.409328 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.409569 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.409684 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:19.412736 (+ 3052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:33471: BlockingRecv error: recv got EOF from 127.18.125.129:33471 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":84,"threads_started":1}
Aug 06 18:35:19 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:19 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:19.641055 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:19.634974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:52555)
0806 18:35:19.635342 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:19.635356 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:19.635462 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:19.635699 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:19.635701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:19.635892 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:19.636018 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:19.636025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.637038 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.637041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:19.637767 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:19.637779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.638247 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.638250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:19.638326 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:19.638931 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:19.638942 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:19.640929 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":115,"threads_started":1}
W20260806 18:35:19.641441 28680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:19.635115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:52555 (local address 127.18.125.190:44671)
0806 18:35:19.635266 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:19.635270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:19.635434 (+ 164us) server_negotiation.cc:407] Connection header received
0806 18:35:19.635534 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:19.635537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:19.635581 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:19.635659 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:19.636289 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.636900 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.637896 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.638129 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.638240 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:19.641336 (+ 3096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:52555: BlockingRecv error: recv got EOF from 127.18.125.130:52555 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 18:35:19 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:19 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:19.850173 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:19.844355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:33077)
0806 18:35:19.844677 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:19.844691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:19.844797 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:19.845155 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:19.845158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:19.845401 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:19.845519 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:19.845527 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.846292 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.846295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:19.846923 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:19.846932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.847386 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.847389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:19.847457 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:19.848069 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:19.848081 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:19.850045 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":103,"threads_started":1}
W20260806 18:35:19.850338 28680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:19.844529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:33077 (local address 127.18.125.190:44671)
0806 18:35:19.844690 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:19.844693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:19.844745 (+ 52us) server_negotiation.cc:407] Connection header received
0806 18:35:19.844930 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:19.844933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:19.844980 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:19.845047 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:19.845640 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.846171 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.847037 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:19.847247 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:19.847324 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:19.850202 (+ 2878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:33077: BlockingRecv error: recv got EOF from 127.18.125.131:33077 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 18:35:20 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:20 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:20.420420 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:20.413587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:34951)
0806 18:35:20.413955 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:20.413969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:20.414111 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:20.414557 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:20.414560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:20.414821 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:20.414935 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:20.414941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.415902 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.415906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:20.417057 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:20.417072 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.417544 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.417548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:20.417622 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:20.418192 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:20.418209 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:20.420288 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":98,"threads_started":1}
W20260806 18:35:20.420779 28684 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:20.413740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:34951 (local address 127.18.125.190:44671)
0806 18:35:20.414064 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:20.414068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:20.414084 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:35:20.414320 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:20.414323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:20.414382 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:20.414489 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:20.415078 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.415744 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.417194 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.417430 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.417514 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:20.420656 (+ 3142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:34951: BlockingRecv error: recv got EOF from 127.18.125.129:34951 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":66,"threads_started":1}
Aug 06 18:35:20 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:20 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:20.648164 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:20.642118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:39091)
0806 18:35:20.642445 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:20.642459 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:20.642549 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:20.642809 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:20.642810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:20.643000 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:20.643098 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:20.643103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.644109 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.644113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:20.645002 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:20.645014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.645475 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.645477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:20.645539 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:20.646118 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:20.646139 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:20.647986 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":120,"threads_started":1}
W20260806 18:35:20.648475 28684 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:20.642356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:39091 (local address 127.18.125.190:44671)
0806 18:35:20.642540 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:20.642544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:20.642563 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:35:20.642617 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:20.642620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:20.642664 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:20.642769 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:20.643229 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.643968 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.645123 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.645365 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.645458 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:20.648199 (+ 2741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:39091: BlockingRecv error: recv got EOF from 127.18.125.130:39091 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 06 18:35:20 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:20 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:20.857005 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:20.851058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:49003)
0806 18:35:20.851446 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:20.851460 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:20.851597 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:20.851878 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:20.851881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:20.852111 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:20.852241 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:20.852247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.853017 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.853022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:20.853577 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:20.853585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.854117 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.854121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:20.854230 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:20.854819 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:20.854831 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:20.856818 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":113,"threads_started":1}
W20260806 18:35:20.857095 28684 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:20.851212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:49003 (local address 127.18.125.190:44671)
0806 18:35:20.851396 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:20.851400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:20.851540 (+ 140us) server_negotiation.cc:407] Connection header received
0806 18:35:20.851700 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:20.851704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:20.851760 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:20.851852 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:20.852360 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.852890 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.853696 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:20.853998 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:20.854111 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:20.856965 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:49003: BlockingRecv error: recv got EOF from 127.18.125.131:49003 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 18:35:21 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:21 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:21.427946 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:21.421366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:55897)
0806 18:35:21.421656 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:21.421670 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:21.421817 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:21.422146 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:21.422148 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:21.422389 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:21.422495 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:21.422501 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.423664 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.423667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:21.424360 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:21.424372 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.424947 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.424950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:21.425025 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:21.425666 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:21.425678 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:21.427813 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":105,"threads_started":1}
W20260806 18:35:21.428517 28688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:21.421415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:55897 (local address 127.18.125.190:44671)
0806 18:35:21.421702 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:21.421706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:21.421719 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:35:21.421913 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:21.421915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:21.421960 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:21.422056 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:21.422743 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.423413 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.424516 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.424799 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.424897 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:21.428403 (+ 3506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:55897: BlockingRecv error: recv got EOF from 127.18.125.129:55897 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
Aug 06 18:35:21 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:21 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:21.657656 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:21.649323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:54525)
0806 18:35:21.649634 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:21.649649 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:21.649758 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:21.650188 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:21.650191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:21.650461 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:21.650594 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:21.650602 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.652095 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.652099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:21.652747 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:21.652759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.653403 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.653409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:21.653653 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:21.654449 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:21.654464 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:21.657439 (+ 2975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":104,"threads_started":1}
W20260806 18:35:21.657778 28688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:21.649435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:54525 (local address 127.18.125.190:44671)
0806 18:35:21.649629 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:21.649633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:21.649723 (+ 90us) server_negotiation.cc:407] Connection header received
0806 18:35:21.649909 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:21.649912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:21.649974 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:21.650084 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:21.650731 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.651669 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.652915 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.653163 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.653275 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:21.657624 (+ 4349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:54525: BlockingRecv error: recv got EOF from 127.18.125.130:54525 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 18:35:21 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:21 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:21.867340 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:21.858053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:52095)
0806 18:35:21.858356 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:21.858373 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:21.858511 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:21.858799 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:21.858802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:21.859017 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:21.859111 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:21.859117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.859890 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.859894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:21.860546 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:21.860555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.861034 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.861037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:21.861122 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:21.865170 (+ 4048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:21.865184 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:21.867202 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":106,"threads_started":1}
W20260806 18:35:21.867558 28688 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:21.858107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:52095 (local address 127.18.125.190:44671)
0806 18:35:21.858269 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:21.858274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:21.858480 (+ 206us) server_negotiation.cc:407] Connection header received
0806 18:35:21.858619 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:21.858625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:21.858690 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:21.858770 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:21.859231 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.859768 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.860667 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:21.860909 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:21.861028 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:21.867412 (+ 6384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:52095: BlockingRecv error: recv got EOF from 127.18.125.131:52095 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 18:35:22 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:22 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:22.434700 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:22.428870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:36281)
0806 18:35:22.429240 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:22.429255 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:22.429364 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:22.429617 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:22.429619 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:22.429808 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:22.429908 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:22.429913 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.430704 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.430707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:22.431277 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:22.431286 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.431794 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.431797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:22.431869 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:22.432515 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:22.432527 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:22.434494 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":101,"threads_started":1}
W20260806 18:35:22.434760 28693 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:22.428975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:36281 (local address 127.18.125.190:44671)
0806 18:35:22.429247 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:22.429251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:22.429335 (+ 84us) server_negotiation.cc:407] Connection header received
0806 18:35:22.429453 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:22.429455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:22.429501 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:22.429580 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:22.430015 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.430567 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.431398 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.431634 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.431716 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:22.434629 (+ 2913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:36281: BlockingRecv error: recv got EOF from 127.18.125.129:36281 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":69,"threads_started":1}
Aug 06 18:35:22 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:22 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:22.668329 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:22.658681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:40275)
0806 18:35:22.659007 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:22.659021 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:22.659125 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:22.659378 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:22.659380 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:22.659580 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:22.659682 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:22.659687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.660460 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.660463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:22.661007 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:22.661015 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.661443 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.661447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:22.661517 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:22.666086 (+ 4569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:22.666100 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:22.668123 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":72,"threads_started":1}
W20260806 18:35:22.668452 28693 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:22.658836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:40275 (local address 127.18.125.190:44671)
0806 18:35:22.659034 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:22.659038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:22.659094 (+ 56us) server_negotiation.cc:407] Connection header received
0806 18:35:22.659210 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:22.659212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:22.659255 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:22.659327 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:22.659789 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.660348 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.661111 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.661318 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.661392 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:22.668340 (+ 6948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:40275: BlockingRecv error: recv got EOF from 127.18.125.130:40275 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 18:35:22 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:22 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:22.874416 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:22.868367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:44409)
0806 18:35:22.868795 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:22.868812 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:22.868932 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:22.869333 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:22.869337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:22.869609 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:22.869734 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:22.869745 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.870483 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.870486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:22.871028 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:22.871035 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.871453 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.871456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:22.871516 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:22.872069 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:22.872081 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:22.874202 (+ 2121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":115,"threads_started":1}
W20260806 18:35:22.874545 28693 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:22.868459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:44409 (local address 127.18.125.190:44671)
0806 18:35:22.868621 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:22.868628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:22.868904 (+ 276us) server_negotiation.cc:407] Connection header received
0806 18:35:22.869065 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:22.869068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:22.869162 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:22.869265 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:22.869857 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.870380 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.871138 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:22.871352 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:22.871431 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:22.874377 (+ 2946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:44409: BlockingRecv error: recv got EOF from 127.18.125.131:44409 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 18:35:23 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:23 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:23.441409 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:23.435511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:36519)
0806 18:35:23.435817 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:23.435831 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:23.435932 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:23.436236 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:23.436238 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:23.436423 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:23.436525 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:23.436530 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.437414 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.437417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:23.437994 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:23.438004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.438491 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.438494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:23.438560 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:23.439081 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:23.439094 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:23.441278 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":117,"threads_started":1}
W20260806 18:35:23.441638 28697 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:23.435664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:36519 (local address 127.18.125.190:44671)
0806 18:35:23.435991 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:23.435995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:23.436008 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:35:23.436046 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:23.436048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:23.436092 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:23.436204 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:23.436634 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.437226 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.438134 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.438380 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.438462 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:23.441464 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:36519: BlockingRecv error: recv got EOF from 127.18.125.129:36519 (error 108)
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":152,"threads_started":1}
Aug 06 18:35:23 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:23 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:23.675806 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:23.669340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:48845)
0806 18:35:23.669625 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:23.669639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:23.669745 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:23.670011 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:23.670012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:23.670199 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:23.670324 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:23.670329 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.671235 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.671238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:23.671818 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:23.671827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.672554 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.672558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:23.672668 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:23.673351 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:23.673364 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:23.675643 (+ 2279us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":97,"threads_started":1}
W20260806 18:35:23.676048 28697 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:23.669421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:48845 (local address 127.18.125.190:44671)
0806 18:35:23.669599 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:23.669602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:23.669715 (+ 113us) server_negotiation.cc:407] Connection header received
0806 18:35:23.669840 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:23.669842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:23.669890 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:23.669970 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:23.670450 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.671120 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.671967 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.672361 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.672502 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:23.675920 (+ 3418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:48845: BlockingRecv error: recv got EOF from 127.18.125.130:48845 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 18:35:23 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:23 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:23.881366 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:23.875424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:46973)
0806 18:35:23.875714 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:23.875728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:23.875839 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:23.876187 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:23.876189 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:23.876376 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:23.876474 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:23.876480 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.877301 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.877304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:23.877858 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:23.877866 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.878366 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.878369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:23.878438 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:23.879110 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:23.879125 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:23.881248 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":111,"threads_started":1}
W20260806 18:35:23.881609 28697 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:23.875484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:46973 (local address 127.18.125.190:44671)
0806 18:35:23.875659 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:23.875662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:23.875799 (+ 137us) server_negotiation.cc:407] Connection header received
0806 18:35:23.875930 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:23.875932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:23.876022 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:23.876108 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:23.876609 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.877181 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.877970 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:23.878188 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:23.878301 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:23.881497 (+ 3196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:46973: BlockingRecv error: recv got EOF from 127.18.125.131:46973 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 18:35:24 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:24 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:24.448453 28700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:24.442352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:60275)
0806 18:35:24.442646 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:24.442661 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:24.442758 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:24.443012 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:24.443014 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:24.443200 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:24.443310 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:24.443316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.444353 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.444356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:24.444935 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:24.444945 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.445458 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.445461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:24.445534 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:24.446147 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:24.446160 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:24.448310 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":101,"threads_started":1}
W20260806 18:35:24.448649 28701 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:24.442422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:60275 (local address 127.18.125.190:44671)
0806 18:35:24.442718 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:24.442722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:24.442737 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:35:24.442845 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:24.442848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:24.442891 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:24.442976 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:24.443429 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.443963 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.445093 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.445322 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.445412 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:24.448472 (+ 3060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:60275: BlockingRecv error: recv got EOF from 127.18.125.129:60275 (error 108)
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":122,"threads_started":1}
Aug 06 18:35:24 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:24 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:24.682904 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:24.676809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:50297)
0806 18:35:24.677127 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:24.677141 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:24.677239 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:24.677505 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:24.677507 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:24.677744 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:24.677852 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:24.677864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.678668 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.678671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:24.679238 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:24.679252 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.679854 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.679859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:24.679943 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:24.680668 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:24.680685 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:24.682767 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":96,"threads_started":1}
W20260806 18:35:24.683074 28701 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:24.676979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:50297 (local address 127.18.125.190:44671)
0806 18:35:24.677165 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:24.677169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:24.677187 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:35:24.677337 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:24.677340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:24.677388 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:24.677468 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:24.677992 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.678550 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.679397 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.679702 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.679817 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:24.682925 (+ 3108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:50297: BlockingRecv error: recv got EOF from 127.18.125.130:50297 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 18:35:24 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:24 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:24.888872 28703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:24.882234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:34657)
0806 18:35:24.882595 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:24.882615 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:24.882742 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:24.883124 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:24.883127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:24.883377 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:24.883517 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:24.883532 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.884546 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.884548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:24.885092 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:24.885100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.885803 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.885808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:24.885922 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:24.886567 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:24.886582 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:24.888739 (+ 2157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":93,"threads_started":1}
W20260806 18:35:24.889087 28701 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:24.882315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:34657 (local address 127.18.125.190:44671)
0806 18:35:24.882697 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:24.882701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:24.882717 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:35:24.882840 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:24.882843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:24.882899 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:24.882989 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:24.883626 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.884444 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.885194 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:24.885517 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:24.885633 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:24.888948 (+ 3315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:34657: BlockingRecv error: recv got EOF from 127.18.125.131:34657 (error 108)
Metrics: {"server-negotiator.queue_time_us":279}
Aug 06 18:35:25 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:25 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:25.455351 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:25.449517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:44969)
0806 18:35:25.449906 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:25.449923 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:25.450055 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:25.450393 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:25.450396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:25.450604 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:25.450722 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:25.450728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.451501 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.451504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:25.452079 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:25.452089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.452570 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.452572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:25.452637 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:25.453132 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:25.453149 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:25.455225 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":95,"threads_started":1}
W20260806 18:35:25.455509 28705 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:25.449577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:44969 (local address 127.18.125.190:44671)
0806 18:35:25.450017 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:25.450024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:25.450040 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:35:25.450162 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:25.450165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:25.450240 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:25.450335 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:25.450833 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.451375 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.452233 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.452470 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.452550 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:25.455373 (+ 2823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:44969: BlockingRecv error: recv got EOF from 127.18.125.129:44969 (error 108)
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":79,"threads_started":1}
Aug 06 18:35:25 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:25 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:25.690122 28706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:25.683835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:35243)
0806 18:35:25.684300 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:25.684324 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:25.684475 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:25.684774 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:25.684777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:25.685004 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:25.685102 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:25.685108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.685904 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.685909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:25.686468 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:25.686476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.686941 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.686944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:25.687034 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:25.687711 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:25.687723 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:25.689940 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":147,"threads_started":1}
W20260806 18:35:25.690163 28705 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:25.684004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:35243 (local address 127.18.125.190:44671)
0806 18:35:25.684267 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:25.684272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:25.684418 (+ 146us) server_negotiation.cc:407] Connection header received
0806 18:35:25.684589 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:25.684594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:25.684652 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:25.684749 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:25.685258 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.685784 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.686595 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.686820 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.686932 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:25.690064 (+ 3132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:35243: BlockingRecv error: recv got EOF from 127.18.125.130:35243 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
Aug 06 18:35:25 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:25 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:25.895818 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:25.889800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:42373)
0806 18:35:25.890187 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:25.890205 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:25.890328 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:25.890599 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:25.890602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:25.890823 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:25.890932 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:25.890938 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.891726 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.891731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:25.892348 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:25.892357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.892847 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.892851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:25.892937 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:25.893483 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:25.893496 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:25.895636 (+ 2140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":146,"threads_started":1}
W20260806 18:35:25.895886 28705 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:25.889975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:42373 (local address 127.18.125.190:44671)
0806 18:35:25.890163 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:25.890167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:25.890293 (+ 126us) server_negotiation.cc:407] Connection header received
0806 18:35:25.890421 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:25.890426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:25.890475 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:25.890575 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:25.891058 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.891612 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.892471 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:25.892716 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:25.892834 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:25.895764 (+ 2930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:42373: BlockingRecv error: recv got EOF from 127.18.125.131:42373 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 18:35:26 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:26 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:26.462357 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:26.456352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:44713)
0806 18:35:26.456827 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:26.456845 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:26.456970 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:26.457350 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:26.457353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:26.457635 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:26.457771 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:26.457784 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.458573 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.458576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:26.459128 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:26.459136 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.459602 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.459605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:26.459667 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:26.460269 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:26.460282 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:26.462214 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":101,"threads_started":1}
W20260806 18:35:26.462541 28710 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:26.456427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:44713 (local address 127.18.125.190:44671)
0806 18:35:26.456719 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:26.456723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:26.456926 (+ 203us) server_negotiation.cc:407] Connection header received
0806 18:35:26.457101 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:26.457104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:26.457171 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:26.457295 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:26.457896 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.458461 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.459256 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.459495 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.459587 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:26.462363 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:44713: BlockingRecv error: recv got EOF from 127.18.125.129:44713 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":71,"threads_started":1}
Aug 06 18:35:26 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:26 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:26.697849 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:26.691155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:41021)
0806 18:35:26.691689 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:26.691713 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:26.691862 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:26.692451 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:26.692454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:26.692689 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:26.692789 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:26.692795 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.693595 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.693601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:26.694167 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:26.694175 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.694683 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.694687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:26.694776 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:26.695439 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:26.695452 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:26.697648 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":128,"threads_started":1}
W20260806 18:35:26.697932 28710 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:26.691424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:41021 (local address 127.18.125.190:44671)
0806 18:35:26.691640 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:26.691645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:26.691798 (+ 153us) server_negotiation.cc:407] Connection header received
0806 18:35:26.691973 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:26.691976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:26.692254 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:26.692431 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:26.692928 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.693477 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.694295 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.694534 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.694672 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:26.697811 (+ 3139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:41021: BlockingRecv error: recv got EOF from 127.18.125.130:41021 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 06 18:35:26 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:26 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:26.902446 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:26.896782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:47037)
0806 18:35:26.897121 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:26.897135 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:26.897247 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:26.897504 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:26.897506 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:26.897699 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:26.897810 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:26.897815 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.898560 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.898563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:26.899114 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:26.899121 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.899569 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.899571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:26.899632 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:26.900254 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:26.900266 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:26.902279 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":116,"threads_started":1}
W20260806 18:35:26.902570 28710 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:26.896847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:47037 (local address 127.18.125.190:44671)
0806 18:35:26.897009 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:26.897015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:26.897224 (+ 209us) server_negotiation.cc:407] Connection header received
0806 18:35:26.897330 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:26.897332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:26.897381 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:26.897458 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:26.897919 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.898457 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.899221 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:26.899465 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:26.899545 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:26.902440 (+ 2895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:47037: BlockingRecv error: recv got EOF from 127.18.125.131:47037 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 18:35:27 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:27 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:27.468891 28713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:27.463209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:42671)
0806 18:35:27.463576 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:27.463590 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:27.463705 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:27.464029 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:27.464031 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:27.464263 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:27.464368 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:27.464373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.465150 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.465152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:27.465725 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:27.465732 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.466172 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.466175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:27.466236 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:27.466761 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:27.466772 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:27.468764 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":104,"threads_started":1}
W20260806 18:35:27.469306 28714 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:27.463344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:42671 (local address 127.18.125.190:44671)
0806 18:35:27.463593 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:27.463597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:27.463672 (+ 75us) server_negotiation.cc:407] Connection header received
0806 18:35:27.463847 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:27.463850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:27.463901 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:27.463988 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:27.464508 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.465053 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.465841 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.466069 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.466148 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:27.468979 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:42671: BlockingRecv error: recv got EOF from 127.18.125.129:42671 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":67,"threads_started":1}
Aug 06 18:35:27 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:27 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:27.708103 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:27.698962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:33865)
0806 18:35:27.699326 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:27.699340 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:27.699479 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:27.699830 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:27.699834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:27.700088 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:27.700261 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:27.700269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.701077 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.701080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:27.701647 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:27.701655 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.702202 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.702205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:27.702308 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:27.705806 (+ 3498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:27.705819 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:27.707970 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":108,"threads_started":1}
W20260806 18:35:27.708438 28714 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:27.699026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:33865 (local address 127.18.125.190:44671)
0806 18:35:27.699162 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:27.699167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:27.699418 (+ 251us) server_negotiation.cc:407] Connection header received
0806 18:35:27.699583 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:27.699587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:27.699663 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:27.699753 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:27.700395 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.700932 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.701843 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.702077 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.702162 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:27.708300 (+ 6138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:33865: BlockingRecv error: recv got EOF from 127.18.125.130:33865 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 18:35:27 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:27 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:27.910171 28716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:27.903501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:52925)
0806 18:35:27.903905 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:27.903923 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:27.904073 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:27.904478 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:27.904481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:27.904680 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:27.904786 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:27.904792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.906031 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.906035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:27.906704 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:27.906714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.907239 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.907243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:27.907321 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:27.907911 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:27.907924 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:27.910041 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":138,"threads_started":1}
W20260806 18:35:27.910356 28714 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:27.903683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:52925 (local address 127.18.125.190:44671)
0806 18:35:27.903905 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:27.903911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:27.904008 (+ 97us) server_negotiation.cc:407] Connection header received
0806 18:35:27.904255 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:27.904259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:27.904321 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:27.904410 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:27.904885 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.905916 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.906870 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:27.907114 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:27.907232 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:27.910203 (+ 2971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:52925: BlockingRecv error: recv got EOF from 127.18.125.131:52925 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 06 18:35:28 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:28 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:28.476740 28717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:28.469927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:45283)
0806 18:35:28.470264 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:28.470281 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:28.470406 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:28.470684 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:28.470686 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:28.470915 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:28.471026 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:28.471032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.471993 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.471997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:28.472974 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:28.472988 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.473593 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.473595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:28.473661 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:28.474211 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:28.474222 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:28.476529 (+ 2307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":124,"threads_started":1}
W20260806 18:35:28.476830 28718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:28.470005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:45283 (local address 127.18.125.190:44671)
0806 18:35:28.470296 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:28.470300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:28.470390 (+ 90us) server_negotiation.cc:407] Connection header received
0806 18:35:28.470499 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:28.470502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:28.470551 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:28.470643 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:28.471155 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.471818 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.473121 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.473484 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.473575 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:28.476682 (+ 3107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:45283: BlockingRecv error: recv got EOF from 127.18.125.129:45283 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":113,"threads_started":1}
Aug 06 18:35:28 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:28 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:28.716228 28719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:28.709264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:46609)
0806 18:35:28.709640 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:28.709666 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:28.709805 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:28.710274 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:28.710277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:28.710491 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:28.710599 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:28.710606 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.711727 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.711731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:28.712709 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:28.712720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.713181 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.713184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:28.713256 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:28.713812 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:28.713825 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:28.716006 (+ 2181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":142,"threads_started":1}
W20260806 18:35:28.716302 28718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:28.709457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:46609 (local address 127.18.125.190:44671)
0806 18:35:28.709662 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:28.709667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:28.709770 (+ 103us) server_negotiation.cc:407] Connection header received
0806 18:35:28.709913 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:28.709916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:28.709975 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:28.710064 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:28.710807 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.711573 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.712822 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.713048 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.713132 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:28.716186 (+ 3054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:46609: BlockingRecv error: recv got EOF from 127.18.125.130:46609 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 18:35:28 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:28 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:28.917248 28721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:28.911157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:48739)
0806 18:35:28.911547 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:28.911565 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:28.911697 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:28.911960 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:28.911962 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:28.912167 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:28.912269 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:28.912275 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.913039 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.913042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:28.913675 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:28.913684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.914155 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.914158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:28.914229 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:28.914882 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:28.914894 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:28.917097 (+ 2203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":187,"threads_started":1}
W20260806 18:35:28.917361 28718 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:28.911263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:48739 (local address 127.18.125.190:44671)
0806 18:35:28.911501 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:28.911510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:28.911651 (+ 141us) server_negotiation.cc:407] Connection header received
0806 18:35:28.911794 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:28.911800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:28.911854 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:28.911928 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:28.912385 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.912917 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.913780 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:28.914016 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:28.914098 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:28.917262 (+ 3164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:48739: BlockingRecv error: recv got EOF from 127.18.125.131:48739 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 18:35:29 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:29 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:29.485565 28722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:29.477865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:46817)
0806 18:35:29.478300 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:29.478321 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:29.478437 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:29.478752 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:29.478755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:29.478962 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:29.479074 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:29.479081 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.480092 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.480102 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:29.480974 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:29.480987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.481611 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.481615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:29.481689 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:29.482603 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:29.482623 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:29.485365 (+ 2742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":113,"threads_started":1}
W20260806 18:35:29.485731 28723 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:29.478044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:46817 (local address 127.18.125.190:44671)
0806 18:35:29.478456 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:29.478460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:29.478477 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:35:29.478524 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:29.478527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:29.478574 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:29.478665 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:29.479219 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.479823 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.481098 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.481451 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.481601 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:29.485586 (+ 3985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:46817: BlockingRecv error: recv got EOF from 127.18.125.129:46817 (error 108)
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":104,"threads_started":1}
Aug 06 18:35:29 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:29 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:29.723078 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:29.717121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:43879)
0806 18:35:29.717424 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:29.717437 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:29.717538 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:29.717783 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:29.717785 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:29.717977 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:29.718076 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:29.718081 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.718852 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.718855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:29.719420 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:29.719428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.719874 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.719877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:29.719941 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:29.720537 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:29.720550 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:29.722916 (+ 2366us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":101,"threads_started":1}
W20260806 18:35:29.723291 28723 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:29.717180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:43879 (local address 127.18.125.190:44671)
0806 18:35:29.717355 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:29.717359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:29.717516 (+ 157us) server_negotiation.cc:407] Connection header received
0806 18:35:29.717621 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:29.717623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:29.717670 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:29.717754 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:29.718187 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.718727 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.719534 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.719769 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.719851 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:29.723111 (+ 3260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:43879: BlockingRecv error: recv got EOF from 127.18.125.130:43879 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 18:35:29 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:29 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.131@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:29.924090 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:29.918283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:44729)
0806 18:35:29.918579 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:29.918593 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:29.918691 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:29.918949 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:29.918951 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:29.919138 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:29.919242 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:29.919247 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.919992 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.919995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:29.920653 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:29.920663 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.921141 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.921143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:29.921208 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:29.921773 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:29.921786 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:29.923954 (+ 2168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":117,"threads_started":1}
W20260806 18:35:29.924364 28723 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:29.918342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:44729 (local address 127.18.125.190:44671)
0806 18:35:29.918478 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:29.918482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:29.918658 (+ 176us) server_negotiation.cc:407] Connection header received
0806 18:35:29.918772 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:29.918774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:29.918821 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:29.918948 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:29.919347 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.919879 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.920787 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:29.921026 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:29.921114 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:29.924187 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.131:44729: BlockingRecv error: recv got EOF from 127.18.125.131:44729 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 18:35:30 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:30 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.129@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:30.493059 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:30.486909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:51973)
0806 18:35:30.487240 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:30.487267 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:30.487380 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:30.487690 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:30.487692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:30.487910 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:30.488024 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:30.488029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:30.488870 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:30.488873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:30.489468 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:30.489478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:30.489974 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:30.489977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:30.490052 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:30.490638 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:30.490651 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:30.492902 (+ 2251us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":95,"threads_started":1}
W20260806 18:35:30.493279 28727 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:30.487076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:51973 (local address 127.18.125.190:44671)
0806 18:35:30.487340 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:30.487344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:30.487358 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:35:30.487509 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:30.487512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:30.487567 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:30.487649 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:30.488176 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:30.488733 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:30.489598 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:30.489842 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:30.489926 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:30.493111 (+ 3185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.129:51973: BlockingRecv error: recv got EOF from 127.18.125.129:51973 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":88,"threads_started":1}
Aug 06 18:35:30 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:35:30 dist-test-slave-8w9v krb5kdc[27608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.125.130@KRBTEST.COM for oryx/127.18.125.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:35:30.730286 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:30.724062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:54665)
0806 18:35:30.724343 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:30.724357 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:30.724487 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:30.724798 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:30.724802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:30.725007 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:30.725106 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:30.725112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:30.725881 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:30.725884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:30.726459 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:30.726468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:30.726952 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:30.726954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:30.727021 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:30.727547 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:30.727559 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:30.730162 (+ 2603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.125.190:44671: Server oryx/127.18.125.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":101,"threads_started":1}
W20260806 18:35:30.730497 28727 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:30.724115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:54665 (local address 127.18.125.190:44671)
0806 18:35:30.724286 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:30.724293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:30.724446 (+ 153us) server_negotiation.cc:407] Connection header received
0806 18:35:30.724605 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:30.724608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:30.724664 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:30.724753 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:30.725215 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:30.725761 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:30.726596 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:30.726837 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:30.726935 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:30.730351 (+ 3416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.125.130:54665: BlockingRecv error: recv got EOF from 127.18.125.130:54665 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:35:30.807989 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 28226
I20260806 18:35:30.816385 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 28389
I20260806 18:35:30.825562 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 28527
I20260806 18:35:30.833189 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 28149
2026-08-06T18:35:30Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18222 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T18:35:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:30Z Disabled control of system clock
I20260806 18:35:30.867179 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:40187
--webserver_interface=127.18.125.190
--webserver_port=0
--builtin_ntp_servers=127.18.125.148:43837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:40187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 18:35:30.988278 28733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:30.988626 28733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:30.988664 28733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:30.993749 28733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:30.993834 28733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:30.993850 28733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:30.993871 28733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:30.993888 28733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:31.000624 28733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:40187
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:40187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:31.002007 28733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:31.003708 28733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:31.013190 28742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:31.013190 28738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:31.013193 28740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:31.013398 28733 server_base.cc:1034] running on GCE node
I20260806 18:35:31.014346 28733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:31.015539 28733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:31.016728 28733 hybrid_clock.cc:648] HybridClock initialized: now 1786041331016693 us; error 51 us; skew 500 ppm
I20260806 18:35:31.019027 28733 webserver.cc:460] Webserver started at http://127.18.125.190:37663/ using document root <none> and password file <none>
I20260806 18:35:31.019712 28733 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:31.019781 28733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:31.020027 28733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:31.022358 28733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "13e9dea7fc8c43ee843a5f574f3fa54d"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "7722308db086c2d0e7475ea6076b0cec"
server_key_iv: "b08e392f7e56b15e5a20052c6d7436de"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.022998 28733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "13e9dea7fc8c43ee843a5f574f3fa54d"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "7722308db086c2d0e7475ea6076b0cec"
server_key_iv: "b08e392f7e56b15e5a20052c6d7436de"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.027226 28733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:35:31.030406 28748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:31.031785 28733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:31.031934 28733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "13e9dea7fc8c43ee843a5f574f3fa54d"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "7722308db086c2d0e7475ea6076b0cec"
server_key_iv: "b08e392f7e56b15e5a20052c6d7436de"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.032069 28733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:31.054237 28733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:31.055065 28733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:31.055290 28733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:31.063128 28802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:40187 every 8 connection(s)
I20260806 18:35:31.063126 28733 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:40187
I20260806 18:35:31.064365 28733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:35:31.067894 28803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:31.074015 28803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d: Bootstrap starting.
I20260806 18:35:31.074460 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 28733
I20260806 18:35:31.074596 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:35:31.074919 18934 external_mini_cluster.cc:1428] Setting key 5d081aa79aace8facd6d748c2d4126c6
I20260806 18:35:31.077339 28803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:31.078182 28803 log.cc:824] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:31.080394 28803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d: No bootstrap required, opened a new log
I20260806 18:35:31.084169 28803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e9dea7fc8c43ee843a5f574f3fa54d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40187 } }
I20260806 18:35:31.084457 28803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:31.084556 28803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13e9dea7fc8c43ee843a5f574f3fa54d, State: Initialized, Role: FOLLOWER
I20260806 18:35:31.085057 28803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e9dea7fc8c43ee843a5f574f3fa54d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40187 } }
I20260806 18:35:31.085240 28803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:35:31.085331 28803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:35:31.085457 28803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:31.085773 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:31.076590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.18.125.190:40187)
0806 18:35:31.077386 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:31.077424 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:31.077493 (+ 69us) server_negotiation.cc:407] Connection header received
0806 18:35:31.078233 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:31.078255 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:31.078787 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:31.079159 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:31.079650 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.080900 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.081885 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.082422 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.082624 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:31.083573 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:31.083578 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:31.083593 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:31.084015 (+ 422us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:31.084074 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:31.084096 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:31.084251 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:31.084439 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:35:31.084766 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":161,"threads_started":1}
I20260806 18:35:31.086776 28803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e9dea7fc8c43ee843a5f574f3fa54d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40187 } }
I20260806 18:35:31.087141 28803 leader_election.cc:302] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13e9dea7fc8c43ee843a5f574f3fa54d; no voters:
I20260806 18:35:31.087484 28803 leader_election.cc:288] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:35:31.087622 28808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:31.087869 28808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [term 1 LEADER]: Becoming Leader. State: Replica: 13e9dea7fc8c43ee843a5f574f3fa54d, State: Running, Role: LEADER
I20260806 18:35:31.088296 28808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e9dea7fc8c43ee843a5f574f3fa54d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40187 } }
I20260806 18:35:31.088922 28803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:31.090353 28809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13e9dea7fc8c43ee843a5f574f3fa54d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e9dea7fc8c43ee843a5f574f3fa54d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40187 } } }
I20260806 18:35:31.090500 28809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:31.090783 28810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13e9dea7fc8c43ee843a5f574f3fa54d. Latest consensus state: current_term: 1 leader_uuid: "13e9dea7fc8c43ee843a5f574f3fa54d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e9dea7fc8c43ee843a5f574f3fa54d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40187 } } }
I20260806 18:35:31.090891 28810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d [sys.catalog]: This master's current role is: LEADER
W20260806 18:35:31.094198 28823 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:35:31.094282 28823 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:35:31.094381 28817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:31.095142 28817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:31.101336 28817 catalog_manager.cc:1347] Generated new cluster ID: 68ebb516d71744e293b02bfd8c565f5e
I20260806 18:35:31.101439 28817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:31.114080 28817 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:35:31.115531 28817 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:31.123778 28817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13e9dea7fc8c43ee843a5f574f3fa54d: Generated new TSK 0
I20260806 18:35:31.124773 28817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:35:31.136793 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40187
--builtin_ntp_servers=127.18.125.148:43837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 18:35:31.259624 28827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:31.259948 28827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:31.259990 28827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:31.264472 28827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:31.264549 28827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:31.264609 28827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:35:31.269587 28827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:31.270756 28827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:31.272219 28827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:31.283128 28837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:31.283139 28834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:31.283139 28835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:31.283529 28827 server_base.cc:1034] running on GCE node
I20260806 18:35:31.283957 28827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:31.284801 28827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:31.286054 28827 hybrid_clock.cc:648] HybridClock initialized: now 1786041331286032 us; error 35 us; skew 500 ppm
I20260806 18:35:31.288314 28827 webserver.cc:460] Webserver started at http://127.18.125.129:39417/ using document root <none> and password file <none>
I20260806 18:35:31.289012 28827 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:31.289113 28827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:31.289361 28827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:31.291781 28827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "6785abfd4aec454fb4ded9ac3d1704be"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "2c4304bbcb8766f80689bf1f45523876"
server_key_iv: "2119701f4651f152ca12d73c1e841f01"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.292426 28827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "6785abfd4aec454fb4ded9ac3d1704be"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "2c4304bbcb8766f80689bf1f45523876"
server_key_iv: "2119701f4651f152ca12d73c1e841f01"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.296620 28827 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:35:31.299386 28843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:31.300879 28827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:31.301041 28827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "6785abfd4aec454fb4ded9ac3d1704be"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "2c4304bbcb8766f80689bf1f45523876"
server_key_iv: "2119701f4651f152ca12d73c1e841f01"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.301184 28827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:31.330220 28827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:31.331099 28827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:31.331352 28827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:31.332051 28827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:31.333177 28827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:31.333268 28827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:31.333351 28827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:31.333398 28827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:31.344548 28827 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:44353
I20260806 18:35:31.344583 28963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:44353 every 8 connection(s)
I20260806 18:35:31.345619 28827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:31.355037 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 28827
I20260806 18:35:31.355144 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:35:31.355528 18934 external_mini_cluster.cc:1428] Setting key 06692e91e1ad4cd22ca395356f78125c
I20260806 18:35:31.356061 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:31.347811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:50079 (local address 127.18.125.190:40187)
0806 18:35:31.347993 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:31.347999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:31.349607 (+ 1608us) server_negotiation.cc:407] Connection header received
0806 18:35:31.351064 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:31.351068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:31.351121 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:31.351208 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:31.352209 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.352756 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.353892 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.354131 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.354213 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:31.355478 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:31.355484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:31.355488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:31.355504 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:31.355528 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:31.355541 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:31.355629 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:31.355822 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 18:35:31.355893 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 18:35:31.357067 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:31.348350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:40187 (local address 127.18.125.129:50079)
0806 18:35:31.349425 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:31.349474 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:31.350830 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:31.351393 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:31.351400 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:31.351488 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:31.351988 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:31.352000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.352880 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.352883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:31.353770 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:31.353780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.354274 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.354277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:31.354434 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:31.355183 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:31.355186 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:31.355242 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:31.355315 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:31.355643 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:31.355654 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:31.355661 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:35:31.355781 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 18:35:31.356078 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":156,"threads_started":1}
I20260806 18:35:31.357687 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40187
--builtin_ntp_servers=127.18.125.148:43837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 18:35:31.358294 28964 heartbeater.cc:347] Connected to a master server at 127.18.125.190:40187
I20260806 18:35:31.358546 28964 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:31.359154 28964 heartbeater.cc:511] Master 127.18.125.190:40187 requested a full tablet report, sending...
I20260806 18:35:31.360824 28767 ts_manager.cc:194] Registered new tserver with Master: 6785abfd4aec454fb4ded9ac3d1704be (127.18.125.129:44353)
I20260806 18:35:31.362066 28767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.129:50079
W20260806 18:35:31.481438 28967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:31.481734 28967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:31.481801 28967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:31.486009 28967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:31.486097 28967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:31.486186 28967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:35:31.491324 28967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:31.492651 28967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:31.494109 28967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:35:31.504621 28967 server_base.cc:1034] running on GCE node
W20260806 18:35:31.504508 28972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:31.504570 28975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:31.504513 28973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:31.505295 28967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:31.506050 28967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:31.507259 28967 hybrid_clock.cc:648] HybridClock initialized: now 1786041331507244 us; error 39 us; skew 500 ppm
I20260806 18:35:31.509388 28967 webserver.cc:460] Webserver started at http://127.18.125.130:40039/ using document root <none> and password file <none>
I20260806 18:35:31.510030 28967 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:31.510128 28967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:31.510387 28967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:31.512677 28967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "406428ba0570460ca48b10f6c996100a"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "ce72fceded9af842eb564f6768c94a8b"
server_key_iv: "376ef87a972754ef532d8fa3f32fc8f7"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.513295 28967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "406428ba0570460ca48b10f6c996100a"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "ce72fceded9af842eb564f6768c94a8b"
server_key_iv: "376ef87a972754ef532d8fa3f32fc8f7"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.517338 28967 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:35:31.520233 28981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:31.521603 28967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:31.521859 28967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "406428ba0570460ca48b10f6c996100a"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "ce72fceded9af842eb564f6768c94a8b"
server_key_iv: "376ef87a972754ef532d8fa3f32fc8f7"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.521997 28967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:31.534125 28967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:31.534973 28967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:31.535247 28967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:31.536188 28967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:31.537916 28967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:31.538009 28967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:31.538085 28967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:31.538143 28967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:31.547663 28967 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:35851
I20260806 18:35:31.547693 29102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:35851 every 8 connection(s)
I20260806 18:35:31.548727 28967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:31.555802 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 28967
I20260806 18:35:31.555931 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:35:31.556308 18934 external_mini_cluster.cc:1428] Setting key e458d6c7c7b0d268c17c654d42e360a1
I20260806 18:35:31.558758 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40187
--builtin_ntp_servers=127.18.125.148:43837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 18:35:31.558987 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:31.550622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:51247 (local address 127.18.125.190:40187)
0806 18:35:31.550809 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:31.550813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:31.551830 (+ 1017us) server_negotiation.cc:407] Connection header received
0806 18:35:31.553305 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:31.553309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:31.553381 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:31.553486 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:31.554405 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.554962 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.556631 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.556855 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.556994 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:31.558366 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:31.558370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:31.558373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:31.558388 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:31.558414 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:31.558427 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:31.558520 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:31.558728 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 18:35:31.558805 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 18:35:31.560837 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:31.551036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:40187 (local address 127.18.125.130:51247)
0806 18:35:31.551681 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:31.551722 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:31.553086 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:31.553662 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:31.553668 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:31.553749 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:31.554247 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:31.554259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.555099 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.555108 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:31.556473 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:31.556486 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.557038 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.557041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:31.557248 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:31.558048 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:31.558050 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:31.558106 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:31.558177 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:31.558555 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:31.558565 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:31.558570 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:35:31.558683 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 18:35:31.559175 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":117,"threads_started":1}
I20260806 18:35:31.562500 29103 heartbeater.cc:347] Connected to a master server at 127.18.125.190:40187
I20260806 18:35:31.562785 29103 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:31.563506 29103 heartbeater.cc:511] Master 127.18.125.190:40187 requested a full tablet report, sending...
I20260806 18:35:31.565062 28767 ts_manager.cc:194] Registered new tserver with Master: 406428ba0570460ca48b10f6c996100a (127.18.125.130:35851)
I20260806 18:35:31.565683 28767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.130:51247
W20260806 18:35:31.680501 29107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:31.680785 29107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:31.680826 29107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:31.685380 29107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:31.685470 29107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:31.685531 29107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:35:31.691231 29107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:31.692497 29107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:31.693962 29107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:31.704362 29112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:31.704362 29113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:31.704356 29115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:31.704628 29107 server_base.cc:1034] running on GCE node
I20260806 18:35:31.705139 29107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:31.705875 29107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:31.707100 29107 hybrid_clock.cc:648] HybridClock initialized: now 1786041331707057 us; error 56 us; skew 500 ppm
I20260806 18:35:31.709228 29107 webserver.cc:460] Webserver started at http://127.18.125.131:40345/ using document root <none> and password file <none>
I20260806 18:35:31.709856 29107 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:31.709921 29107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:31.710182 29107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:31.712486 29107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "4f77b73f98fa4161abbd8bc7ac9a6f3b"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "5502ac3037ffdb8b9d418996aa6bf8a7"
server_key_iv: "af592b50a27db6da9605226e1a74ab2a"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.713075 29107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "4f77b73f98fa4161abbd8bc7ac9a6f3b"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "5502ac3037ffdb8b9d418996aa6bf8a7"
server_key_iv: "af592b50a27db6da9605226e1a74ab2a"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.717200 29107 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:35:31.720258 29121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:31.721604 29107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:31.721755 29107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "4f77b73f98fa4161abbd8bc7ac9a6f3b"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "5502ac3037ffdb8b9d418996aa6bf8a7"
server_key_iv: "af592b50a27db6da9605226e1a74ab2a"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.721890 29107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:31.732803 29107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:31.733575 29107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:31.733788 29107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:31.734416 29107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:31.735469 29107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:31.735527 29107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:31.735570 29107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:31.735585 29107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:31.745229 29107 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:34323
I20260806 18:35:31.745281 29242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:34323 every 8 connection(s)
I20260806 18:35:31.746713 29107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:31.747494 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 29107
I20260806 18:35:31.747604 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:35:31.747956 18934 external_mini_cluster.cc:1428] Setting key 7f28861a1dd5f1a1b76ba3bc8041d28d
I20260806 18:35:31.758821 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:31.750715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:38363 (local address 127.18.125.190:40187)
0806 18:35:31.750845 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:31.750849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:31.750864 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:35:31.752390 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:31.752394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:31.752470 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:31.752577 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:31.753644 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.754459 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.756473 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.756828 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.756943 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:31.758235 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:31.758238 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:31.758241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:31.758258 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:31.758288 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:31.758297 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:31.758380 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:31.758625 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 18:35:31.758691 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:35:31.759788 29245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:31.749560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:40187 (local address 127.18.125.131:38363)
0806 18:35:31.750447 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:31.750492 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:31.752087 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:31.752766 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:31.752775 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:31.752863 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:31.753472 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:31.753491 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.754611 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.754615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:31.756300 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:31.756313 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:31.757009 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:31.757014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:31.757202 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:31.757944 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:31.757946 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:31.757991 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:31.758063 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:31.758383 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:31.758397 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:31.758404 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:35:31.758570 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 18:35:31.758839 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":146,"threads_started":1}
I20260806 18:35:31.761761 29243 heartbeater.cc:347] Connected to a master server at 127.18.125.190:40187
I20260806 18:35:31.762058 29243 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:31.762739 29243 heartbeater.cc:511] Master 127.18.125.190:40187 requested a full tablet report, sending...
I20260806 18:35:31.763909 28767 ts_manager.cc:194] Registered new tserver with Master: 4f77b73f98fa4161abbd8bc7ac9a6f3b (127.18.125.131:34323)
I20260806 18:35:31.764570 28767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.131:38363
I20260806 18:35:31.775691 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:35:31.780376 28806 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:31.778619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.18.125.190:40187)
0806 18:35:31.778803 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:31.778810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:31.779167 (+ 357us) server_negotiation.cc:407] Connection header received
0806 18:35:31.779238 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:31.779242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:31.779353 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:31.779469 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:31.779901 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43416: BlockingRecv error: recv got EOF from 127.0.0.1:43416 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:35:31.780560 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 28827
I20260806 18:35:31.790158 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 28967
I20260806 18:35:31.801509 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 29107
I20260806 18:35:31.811162 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 28733
2026-08-06T18:35:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (977 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T18:35:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:31Z Disabled control of system clock
I20260806 18:35:31.846917 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:37901
--webserver_interface=127.18.125.190
--webserver_port=0
--builtin_ntp_servers=127.18.125.148:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:37901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:35:31.965196 29254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:31.965560 29254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:31.965642 29254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:31.969960 29254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:31.970077 29254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:31.970105 29254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:31.970145 29254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:31.970180 29254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:31.975240 29254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:37901
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:37901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:31.976609 29254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:31.978103 29254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:35:31.987576 29254 server_base.cc:1034] running on GCE node
W20260806 18:35:31.987514 29260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:31.987496 29259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:31.987515 29262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:31.988513 29254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:31.989714 29254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:31.990916 29254 hybrid_clock.cc:648] HybridClock initialized: now 1786041331990898 us; error 34 us; skew 500 ppm
I20260806 18:35:31.993108 29254 webserver.cc:460] Webserver started at http://127.18.125.190:41015/ using document root <none> and password file <none>
I20260806 18:35:31.993719 29254 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:31.993815 29254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:31.994060 29254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:31.996279 29254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "b4cae21702d74b70bdbc2ef3547a9e3e"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "80ccf31908e71d7480b78342c5f08db4"
server_key_iv: "cf0283593297b5ec835c5e9f22a35005"
server_key_version: "encryptionkey@0"
I20260806 18:35:31.996873 29254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "b4cae21702d74b70bdbc2ef3547a9e3e"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "80ccf31908e71d7480b78342c5f08db4"
server_key_iv: "cf0283593297b5ec835c5e9f22a35005"
server_key_version: "encryptionkey@0"
I20260806 18:35:32.001228 29254 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:35:32.003892 29268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:32.005693 29254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:35:32.005851 29254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "b4cae21702d74b70bdbc2ef3547a9e3e"
format_stamp: "Formatted at 2026-08-06 18:35:31 on dist-test-slave-8w9v"
server_key: "80ccf31908e71d7480b78342c5f08db4"
server_key_iv: "cf0283593297b5ec835c5e9f22a35005"
server_key_version: "encryptionkey@0"
I20260806 18:35:32.006007 29254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:32.013319 29254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:32.014199 29254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:32.014402 29254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:32.024040 29254 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:37901
I20260806 18:35:32.024060 29323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:37901 every 8 connection(s)
I20260806 18:35:32.025530 29254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:35:32.030664 29324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:32.034210 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 29254
I20260806 18:35:32.034386 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:35:32.035092 18934 external_mini_cluster.cc:1428] Setting key aae6d93322cd375eaa9da968efdaa79e
I20260806 18:35:32.038738 29324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e: Bootstrap starting.
I20260806 18:35:32.042303 29324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:32.043421 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:32.037476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44094 (local address 127.18.125.190:37901)
0806 18:35:32.038365 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:32.038389 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:32.038438 (+ 49us) server_negotiation.cc:407] Connection header received
0806 18:35:32.039612 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:32.039640 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:32.040377 (+ 737us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:32.040880 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:32.041721 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:32.041727 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:32.041745 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:32.041757 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:32.041834 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:32.041859 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:32.041973 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:32.042084 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 18:35:32.042371 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":173,"threads_started":1}
I20260806 18:35:32.043418 29324 log.cc:824] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:32.046262 29324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e: No bootstrap required, opened a new log
I20260806 18:35:32.052959 29324 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cae21702d74b70bdbc2ef3547a9e3e" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37901 } }
I20260806 18:35:32.053422 29324 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:32.053505 29324 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4cae21702d74b70bdbc2ef3547a9e3e, State: Initialized, Role: FOLLOWER
I20260806 18:35:32.054134 29324 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cae21702d74b70bdbc2ef3547a9e3e" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37901 } }
I20260806 18:35:32.054350 29324 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:35:32.054462 29324 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:35:32.054608 29324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:32.056474 29324 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cae21702d74b70bdbc2ef3547a9e3e" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37901 } }
I20260806 18:35:32.056999 29324 leader_election.cc:302] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4cae21702d74b70bdbc2ef3547a9e3e; no voters:
I20260806 18:35:32.057526 29324 leader_election.cc:288] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:35:32.057655 29329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:32.057948 29329 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [term 1 LEADER]: Becoming Leader. State: Replica: b4cae21702d74b70bdbc2ef3547a9e3e, State: Running, Role: LEADER
I20260806 18:35:32.058311 29329 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cae21702d74b70bdbc2ef3547a9e3e" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37901 } }
I20260806 18:35:32.059326 29324 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:32.060253 29331 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4cae21702d74b70bdbc2ef3547a9e3e. Latest consensus state: current_term: 1 leader_uuid: "b4cae21702d74b70bdbc2ef3547a9e3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cae21702d74b70bdbc2ef3547a9e3e" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37901 } } }
I20260806 18:35:32.060403 29331 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:32.060235 29330 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4cae21702d74b70bdbc2ef3547a9e3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4cae21702d74b70bdbc2ef3547a9e3e" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37901 } } }
I20260806 18:35:32.061035 29330 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e [sys.catalog]: This master's current role is: LEADER
W20260806 18:35:32.065289 29345 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:35:32.065413 29345 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:35:32.065495 29337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:32.066392 29337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:32.072559 29337 catalog_manager.cc:1347] Generated new cluster ID: d5ffebfaef4b46b99d6740ed3732eaaf
I20260806 18:35:32.072650 29337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:32.083643 29337 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:35:32.084038 29337 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:32.091202 29337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4cae21702d74b70bdbc2ef3547a9e3e: Generated new TSK 0
I20260806 18:35:32.092005 29337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:35:32.104018 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37901
--builtin_ntp_servers=127.18.125.148:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:35:32.219172 29350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:32.219462 29350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:32.219503 29350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:32.223606 29350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:32.223681 29350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:32.223778 29350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:35:32.228708 29350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:32.229908 29350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:32.231393 29350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:32.242718 29358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:32.242718 29355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:32.242743 29356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:32.242950 29350 server_base.cc:1034] running on GCE node
I20260806 18:35:32.244269 29350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:32.245108 29350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:32.246326 29350 hybrid_clock.cc:648] HybridClock initialized: now 1786041332246293 us; error 53 us; skew 500 ppm
I20260806 18:35:32.248900 29350 webserver.cc:460] Webserver started at http://127.18.125.129:38095/ using document root <none> and password file <none>
I20260806 18:35:32.249614 29350 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:32.249724 29350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:32.249995 29350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:32.252660 29350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "824dc04dd3f345be9d37c89e11817c0b"
format_stamp: "Formatted at 2026-08-06 18:35:32 on dist-test-slave-8w9v"
server_key: "13f4764305108a97ae78f41467a1eac5"
server_key_iv: "b42fd82b49f1a496d0404fc1cb617639"
server_key_version: "encryptionkey@0"
I20260806 18:35:32.253345 29350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "824dc04dd3f345be9d37c89e11817c0b"
format_stamp: "Formatted at 2026-08-06 18:35:32 on dist-test-slave-8w9v"
server_key: "13f4764305108a97ae78f41467a1eac5"
server_key_iv: "b42fd82b49f1a496d0404fc1cb617639"
server_key_version: "encryptionkey@0"
I20260806 18:35:32.258400 29350 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 18:35:32.261924 29364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:32.263615 29350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:35:32.263808 29350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "824dc04dd3f345be9d37c89e11817c0b"
format_stamp: "Formatted at 2026-08-06 18:35:32 on dist-test-slave-8w9v"
server_key: "13f4764305108a97ae78f41467a1eac5"
server_key_iv: "b42fd82b49f1a496d0404fc1cb617639"
server_key_version: "encryptionkey@0"
I20260806 18:35:32.263965 29350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:32.269214 29350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:32.269954 29350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:32.270183 29350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:32.270813 29350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:32.271845 29350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:32.271927 29350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:32.272022 29350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:32.272075 29350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:32.281708 29350 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:41913
I20260806 18:35:32.281800 29479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:41913 every 8 connection(s)
I20260806 18:35:32.282739 29350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:32.289633 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:32.284787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:60763 (local address 127.18.125.190:37901)
0806 18:35:32.284943 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:32.284947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:32.286097 (+ 1150us) server_negotiation.cc:407] Connection header received
0806 18:35:32.287534 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:32.287538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:32.287591 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:32.287702 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:32.289038 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:32.289042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:32.289045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:32.289068 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:32.289092 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:32.289101 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:32.289215 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:32.289379 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 18:35:32.289467 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:35:32.290588 29482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:32.285161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:37901 (local address 127.18.125.129:60763)
0806 18:35:32.285927 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:32.285971 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:32.287304 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:32.287889 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:32.287897 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:32.287991 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:32.288710 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:32.288712 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:32.288755 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:32.288833 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:32.289208 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:32.289219 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:32.289227 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 18:35:32.289427 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 18:35:32.289703 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":139,"threads_started":1}
I20260806 18:35:32.291700 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 29350
I20260806 18:35:32.291795 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:35:32.292085 29480 heartbeater.cc:347] Connected to a master server at 127.18.125.190:37901
I20260806 18:35:32.292186 18934 external_mini_cluster.cc:1428] Setting key 39de5c692f3aa0bd8452de3e4d8bc0ef
I20260806 18:35:32.292402 29480 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:32.292944 29480 heartbeater.cc:511] Master 127.18.125.190:37901 requested a full tablet report, sending...
I20260806 18:35:32.294417 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37901
--builtin_ntp_servers=127.18.125.148:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 18:35:32.294777 29285 ts_manager.cc:194] Registered new tserver with Master: 824dc04dd3f345be9d37c89e11817c0b (127.18.125.129:41913)
W20260806 18:35:32.412376 29483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:32.412662 29483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:32.412706 29483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:32.417016 29483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:32.417092 29483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:32.417155 29483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:35:32.422112 29483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:32.423259 29483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:32.424669 29483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:32.434803 29492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:32.434832 29489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:32.434832 29490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:32.435101 29483 server_base.cc:1034] running on GCE node
I20260806 18:35:32.435590 29483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:32.436474 29483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:32.437677 29483 hybrid_clock.cc:648] HybridClock initialized: now 1786041332437640 us; error 52 us; skew 500 ppm
I20260806 18:35:32.439731 29483 webserver.cc:460] Webserver started at http://127.18.125.130:34793/ using document root <none> and password file <none>
I20260806 18:35:32.440438 29483 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:32.440510 29483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:32.440761 29483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:32.442943 29483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "3c19df197aff4d61bfaa6712a8b98d9d"
format_stamp: "Formatted at 2026-08-06 18:35:32 on dist-test-slave-8w9v"
server_key: "0c2513bdca16a08164f8ef5012c51c3f"
server_key_iv: "bf9982da0865444af0135e9ae81435ce"
server_key_version: "encryptionkey@0"
I20260806 18:35:32.443562 29483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "3c19df197aff4d61bfaa6712a8b98d9d"
format_stamp: "Formatted at 2026-08-06 18:35:32 on dist-test-slave-8w9v"
server_key: "0c2513bdca16a08164f8ef5012c51c3f"
server_key_iv: "bf9982da0865444af0135e9ae81435ce"
server_key_version: "encryptionkey@0"
I20260806 18:35:32.447736 29483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:35:32.450884 29498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:32.452368 29483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:35:32.452520 29483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "3c19df197aff4d61bfaa6712a8b98d9d"
format_stamp: "Formatted at 2026-08-06 18:35:32 on dist-test-slave-8w9v"
server_key: "0c2513bdca16a08164f8ef5012c51c3f"
server_key_iv: "bf9982da0865444af0135e9ae81435ce"
server_key_version: "encryptionkey@0"
I20260806 18:35:32.452661 29483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:32.457638 29483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:32.458371 29483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:32.458583 29483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:32.459224 29483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:32.460430 29483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:32.460525 29483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:32.460587 29483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:32.460620 29483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:32.471596 29483 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:43359
I20260806 18:35:32.471678 29615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:43359 every 8 connection(s)
I20260806 18:35:32.472719 29483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:32.479157 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:32.474817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:60535 (local address 127.18.125.190:37901)
0806 18:35:32.475009 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:32.475015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:32.476106 (+ 1091us) server_negotiation.cc:407] Connection header received
0806 18:35:32.477551 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:32.477554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:32.477593 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:32.477672 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:32.478746 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:32.478749 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:32.478751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:32.478763 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:32.478782 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:32.478790 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:32.478885 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:32.479001 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 18:35:32.479043 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 18:35:32.480019 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:32.475174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:37901 (local address 127.18.125.130:60535)
0806 18:35:32.475940 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:32.475996 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:32.477348 (+ 1352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:32.477837 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:32.477844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:32.477928 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:32.478485 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:32.478487 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:32.478527 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:32.478595 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:32.478872 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:32.478880 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:32.478884 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:35:32.479018 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 18:35:32.479206 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":168,"threads_started":1}
I20260806 18:35:32.481330 29616 heartbeater.cc:347] Connected to a master server at 127.18.125.190:37901
I20260806 18:35:32.481616 29616 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:32.482167 29616 heartbeater.cc:511] Master 127.18.125.190:37901 requested a full tablet report, sending...
I20260806 18:35:32.482312 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 29483
I20260806 18:35:32.482435 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:35:32.482829 18934 external_mini_cluster.cc:1428] Setting key 260f3997e03c8aab4ed2c57a38ef3615
I20260806 18:35:32.483464 29285 ts_manager.cc:194] Registered new tserver with Master: 3c19df197aff4d61bfaa6712a8b98d9d (127.18.125.130:43359)
I20260806 18:35:32.485145 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37901
--builtin_ntp_servers=127.18.125.148:33399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:35:32.602948 29619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:32.603256 29619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:32.603328 29619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:32.607614 29619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:32.607719 29619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:32.607822 29619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:35:32.613193 29619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:32.614432 29619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:32.615972 29619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:32.626956 29624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:32.626957 29625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:32.626948 29627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:32.627324 29619 server_base.cc:1034] running on GCE node
I20260806 18:35:32.627840 29619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:32.628660 29619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:32.629877 29619 hybrid_clock.cc:648] HybridClock initialized: now 1786041332629861 us; error 34 us; skew 500 ppm
I20260806 18:35:32.632107 29619 webserver.cc:460] Webserver started at http://127.18.125.131:35657/ using document root <none> and password file <none>
I20260806 18:35:32.632805 29619 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:32.632907 29619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:32.633149 29619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:32.635437 29619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "21f0e482f15d4966b311c819a383fe0b"
format_stamp: "Formatted at 2026-08-06 18:35:32 on dist-test-slave-8w9v"
server_key: "b5fc140b17856367e3d30488f5303d65"
server_key_iv: "3c86a6aa71d642830db5254b245ccdd9"
server_key_version: "encryptionkey@0"
I20260806 18:35:32.636060 29619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "21f0e482f15d4966b311c819a383fe0b"
format_stamp: "Formatted at 2026-08-06 18:35:32 on dist-test-slave-8w9v"
server_key: "b5fc140b17856367e3d30488f5303d65"
server_key_iv: "3c86a6aa71d642830db5254b245ccdd9"
server_key_version: "encryptionkey@0"
I20260806 18:35:32.641227 29619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 18:35:32.644304 29633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:32.645757 29619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:35:32.645925 29619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "21f0e482f15d4966b311c819a383fe0b"
format_stamp: "Formatted at 2026-08-06 18:35:32 on dist-test-slave-8w9v"
server_key: "b5fc140b17856367e3d30488f5303d65"
server_key_iv: "3c86a6aa71d642830db5254b245ccdd9"
server_key_version: "encryptionkey@0"
I20260806 18:35:32.646073 29619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:32.650965 29619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:32.651610 29619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:32.651816 29619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:32.652498 29619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:32.653460 29619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:32.653532 29619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:32.653602 29619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:32.653656 29619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:32.663712 29619 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:43413
I20260806 18:35:32.663733 29753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:43413 every 8 connection(s)
I20260806 18:35:32.664778 29619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:32.671341 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:32.666860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:36385 (local address 127.18.125.190:37901)
0806 18:35:32.667028 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:32.667032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:32.668034 (+ 1002us) server_negotiation.cc:407] Connection header received
0806 18:35:32.669519 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:32.669523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:32.669574 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:32.669692 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:32.670810 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:32.670814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:32.670817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:35:32.670829 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:32.670858 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:35:32.670867 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:32.670999 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:32.671146 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 18:35:32.671198 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:35:32.672331 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:32.667191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:37901 (local address 127.18.125.131:36385)
0806 18:35:32.667874 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:32.667915 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:32.669274 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:32.669869 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:32.669876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:32.669965 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:32.670545 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:35:32.670547 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:32.670589 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:35:32.670656 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:32.670972 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:32.670982 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:32.670989 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:35:32.671178 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 18:35:32.671419 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":117,"threads_started":1}
I20260806 18:35:32.672434 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 29619
I20260806 18:35:32.672535 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:35:32.672849 18934 external_mini_cluster.cc:1428] Setting key 9fd63e213daf494dc9f92ea2df1a174f
I20260806 18:35:32.673643 29754 heartbeater.cc:347] Connected to a master server at 127.18.125.190:37901
I20260806 18:35:32.673919 29754 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:32.674397 29754 heartbeater.cc:511] Master 127.18.125.190:37901 requested a full tablet report, sending...
I20260806 18:35:32.675541 29284 ts_manager.cc:194] Registered new tserver with Master: 21f0e482f15d4966b311c819a383fe0b (127.18.125.131:43413)
I20260806 18:35:32.687269 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:35:32.690363 29327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:35:32.689370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44106 (local address 127.18.125.190:37901)
0806 18:35:32.689516 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:32.689523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:32.689701 (+ 178us) server_negotiation.cc:407] Connection header received
0806 18:35:32.689878 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:32.689881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:32.689918 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:32.690001 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:32.690249 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44106: BlockingRecv error: recv got EOF from 127.0.0.1:44106 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:35:32.690913 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 29350
I20260806 18:35:32.698035 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 29483
I20260806 18:35:32.705632 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 29619
I20260806 18:35:32.713848 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 29254
2026-08-06T18:35:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (901 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:35:32 dist-test-slave-8w9v krb5kdc[29764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:35:32 dist-test-slave-8w9v krb5kdc[29764](info): set up 2 sockets
Aug 06 18:35:32 dist-test-slave-8w9v krb5kdc[29764](info): commencing operation
krb5kdc: starting...
W20260806 18:35:33.808872 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:35:33 dist-test-slave-8w9v krb5kdc[29764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:35:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:35:33.945484 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:37341
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:37341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:34.069134 29777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:34.069458 29777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:34.069550 29777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:34.073868 29777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:35:34.073939 29777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:34.073962 29777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:34.073980 29777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:35:34.074023 29777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:35:34.079869 29777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:37341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:37341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:34.081394 29777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:34.082499 29777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:34.092352 29782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:34.092343 29783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:34.092343 29785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:34.092573 29777 server_base.cc:1034] running on GCE node
I20260806 18:35:34.093209 29777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:34.094456 29777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:34.095682 29777 hybrid_clock.cc:648] HybridClock initialized: now 1786041334095659 us; error 34 us; skew 500 ppm
Aug 06 18:35:34 dist-test-slave-8w9v krb5kdc[29764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041334, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:34.098969 29777 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:35:34.099951 29777 webserver.cc:460] Webserver started at http://127.18.125.190:36971/ using document root <none> and password file <none>
I20260806 18:35:34.100571 29777 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:34.100637 29777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:34.100813 29777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:34.102998 29777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "1153529a57fe4f4e8a880db64f3632f0"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "c3217546e1c04fa469f6d541ae77a74e"
server_key_iv: "13024e8bb773b79ce8d9a8fcef2bde06"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.103562 29777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "1153529a57fe4f4e8a880db64f3632f0"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "c3217546e1c04fa469f6d541ae77a74e"
server_key_iv: "13024e8bb773b79ce8d9a8fcef2bde06"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.107733 29777 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:35:34.110635 29792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:34.112051 29777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:34.112260 29777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "1153529a57fe4f4e8a880db64f3632f0"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "c3217546e1c04fa469f6d541ae77a74e"
server_key_iv: "13024e8bb773b79ce8d9a8fcef2bde06"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.112386 29777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:34.128786 29777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:34.132202 29777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:34.132419 29777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:34.140380 29777 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:37341
I20260806 18:35:34.140698 29847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:37341 every 8 connection(s)
I20260806 18:35:34.141469 29777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:35:34.144164 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 29777
I20260806 18:35:34.144294 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:35:34.144654 18934 external_mini_cluster.cc:1428] Setting key e90b5f6ccbea658e43dcff6b845d8d64
I20260806 18:35:34.144749 29848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:34.151597 29848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0: Bootstrap starting.
Aug 06 18:35:34 dist-test-slave-8w9v krb5kdc[29764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:35:34.154363 29848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:34.155258 29848 log.cc:824] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:34.157850 29848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0: No bootstrap required, opened a new log
I20260806 18:35:34.160509 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.146209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44516 (local address 127.18.125.190:37341)
0806 18:35:34.146680 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:34.146697 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:34.146732 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:35:34.147644 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:34.147670 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:34.148009 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:34.148448 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:34.149343 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.150342 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.151201 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.151730 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.151924 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.154853 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:34.154860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:34.154875 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:34.154905 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:34.157007 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:34.157549 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:34.157554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:34.157558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:34.157693 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:34.158048 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:34.158052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:34.158053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:34.158592 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:34.158833 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:34.159103 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 18:35:34.159415 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":146,"threads_started":1}
I20260806 18:35:34.162575 29848 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1153529a57fe4f4e8a880db64f3632f0" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37341 } }
I20260806 18:35:34.162904 29848 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:34.162966 29848 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1153529a57fe4f4e8a880db64f3632f0, State: Initialized, Role: FOLLOWER
I20260806 18:35:34.163453 29848 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1153529a57fe4f4e8a880db64f3632f0" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37341 } }
I20260806 18:35:34.163622 29848 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:35:34.163713 29848 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:35:34.163843 29848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:34.164958 29848 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1153529a57fe4f4e8a880db64f3632f0" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37341 } }
I20260806 18:35:34.165345 29848 leader_election.cc:302] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1153529a57fe4f4e8a880db64f3632f0; no voters:
I20260806 18:35:34.165728 29848 leader_election.cc:288] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:35:34.165845 29853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:34.166123 29853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [term 1 LEADER]: Becoming Leader. State: Replica: 1153529a57fe4f4e8a880db64f3632f0, State: Running, Role: LEADER
I20260806 18:35:34.166462 29853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1153529a57fe4f4e8a880db64f3632f0" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37341 } }
I20260806 18:35:34.167248 29848 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:35:34.168521 29855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1153529a57fe4f4e8a880db64f3632f0. Latest consensus state: current_term: 1 leader_uuid: "1153529a57fe4f4e8a880db64f3632f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1153529a57fe4f4e8a880db64f3632f0" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37341 } } }
I20260806 18:35:34.168511 29854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1153529a57fe4f4e8a880db64f3632f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1153529a57fe4f4e8a880db64f3632f0" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 37341 } } }
I20260806 18:35:34.168813 29854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:35:34.168852 29855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0 [sys.catalog]: This master's current role is: LEADER
W20260806 18:35:34.172829 29868 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:35:34.172930 29868 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:35:34.173004 29863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:35:34.173724 29863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:35:34.179765 29863 catalog_manager.cc:1347] Generated new cluster ID: ae11602226db42b0a08a0fcdf089839f
I20260806 18:35:34.179850 29863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:35:34.188045 29863 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:35:34.189082 29863 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:35:34.194257 29863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0: Generated new TSK 0
I20260806 18:35:34.195024 29863 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:35:34.228013 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37341
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:34.360637 29875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:34.360931 29875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:34.361016 29875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:34.365252 29875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:34.365325 29875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:34.365425 29875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:35:34.371372 29875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:34.372596 29875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:34.373675 29875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:34.383975 29880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:34.384097 29881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:34.383977 29883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:34.385061 29875 server_base.cc:1034] running on GCE node
I20260806 18:35:34.385516 29875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:34.386138 29875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:34.387341 29875 hybrid_clock.cc:648] HybridClock initialized: now 1786041334387320 us; error 39 us; skew 500 ppm
Aug 06 18:35:34 dist-test-slave-8w9v krb5kdc[29764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041334, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:34.390285 29875 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:35:34.391249 29875 webserver.cc:460] Webserver started at http://127.18.125.129:44139/ using document root <none> and password file <none>
I20260806 18:35:34.391845 29875 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:34.391909 29875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:34.392194 29875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:34.394325 29875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "607d44838e334715b56165a350a44006"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "62591abbd6637bc28a712be44bb24ec4"
server_key_iv: "8ca7c18b1402bebcb58ec36808fcabbf"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.394939 29875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "607d44838e334715b56165a350a44006"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "62591abbd6637bc28a712be44bb24ec4"
server_key_iv: "8ca7c18b1402bebcb58ec36808fcabbf"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.399524 29875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 18:35:34.402530 29890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:34.403999 29875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:35:34.404197 29875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "607d44838e334715b56165a350a44006"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "62591abbd6637bc28a712be44bb24ec4"
server_key_iv: "8ca7c18b1402bebcb58ec36808fcabbf"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.404338 29875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:34.414558 29875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:34.418463 29875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:34.418697 29875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:34.419374 29875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:34.420359 29875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:34.420442 29875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:34.420542 29875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:34.420594 29875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:34.431764 29875 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:38535
I20260806 18:35:34.431869 30004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:38535 every 8 connection(s)
I20260806 18:35:34.432906 29875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:35:34.437018 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 29875
I20260806 18:35:34.437170 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:35:34.437443 18934 external_mini_cluster.cc:1428] Setting key 48733091fc4951e8a05b01ce619864ee
Aug 06 18:35:34 dist-test-slave-8w9v krb5kdc[29764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041334, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:35:34.450110 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.435144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:54567 (local address 127.18.125.190:37341)
0806 18:35:34.435319 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:34.435325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:34.436610 (+ 1285us) server_negotiation.cc:407] Connection header received
0806 18:35:34.438347 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:34.438350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:34.438395 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:34.438463 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:34.439927 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.440851 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.442719 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.443149 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.443282 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.446226 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:34.446230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:34.446232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:34.446260 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:34.447793 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:34.448465 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:34.448471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:34.448473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:34.448556 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:34.448922 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:34.448925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:34.448927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:34.449126 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:34.449237 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:34.449799 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 18:35:34.449935 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 18:35:34.450989 30007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.435586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:37341 (local address 127.18.125.129:54567)
0806 18:35:34.436428 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:34.436478 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:34.438084 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:34.438657 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:34.438667 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:34.439205 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:34.439736 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.439751 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.441044 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.441050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.442571 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.442585 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.443335 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.443339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.443497 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.444243 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:34.444268 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:34.445931 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:34.447942 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:34.447949 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:34.447969 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:34.448350 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:34.448657 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:34.448660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:34.448662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:34.448813 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:34.449238 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:34.449246 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:34.449528 (+ 282us) client_negotiation.cc:769] Sending connection context
0806 18:35:34.449732 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 18:35:34.450104 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":136,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
I20260806 18:35:34.452402 30005 heartbeater.cc:347] Connected to a master server at 127.18.125.190:37341
I20260806 18:35:34.452695 30005 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:34.453280 30005 heartbeater.cc:511] Master 127.18.125.190:37341 requested a full tablet report, sending...
I20260806 18:35:34.455168 29810 ts_manager.cc:194] Registered new tserver with Master: 607d44838e334715b56165a350a44006 (127.18.125.129:38535)
I20260806 18:35:34.456950 29810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:54567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:35:34.472967 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37341
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:34.593397 30010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:34.593688 30010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:34.593737 30010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:34.597877 30010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:34.597946 30010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:34.598011 30010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:35:34.603472 30010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:34.604645 30010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:34.605659 30010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:34.615828 30017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:34.615789 30019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:34.615804 30016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:34.616169 30010 server_base.cc:1034] running on GCE node
I20260806 18:35:34.616788 30010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:34.617551 30010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:34.618760 30010 hybrid_clock.cc:648] HybridClock initialized: now 1786041334618744 us; error 32 us; skew 500 ppm
Aug 06 18:35:34 dist-test-slave-8w9v krb5kdc[29764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041334, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:34.621820 30010 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:35:34.622757 30010 webserver.cc:460] Webserver started at http://127.18.125.130:43501/ using document root <none> and password file <none>
I20260806 18:35:34.623359 30010 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:34.623464 30010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:34.623716 30010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:34.625965 30010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "e9194a499fee41c492e2294e5051fbe7"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "74a30d4e746076e097a79df6eb25c623"
server_key_iv: "5a15e0c9cf2f56ee8c0255a32b503a4e"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.626592 30010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "e9194a499fee41c492e2294e5051fbe7"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "74a30d4e746076e097a79df6eb25c623"
server_key_iv: "5a15e0c9cf2f56ee8c0255a32b503a4e"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.630764 30010 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:35:34.633617 30026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:34.634899 30010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:35:34.635057 30010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "e9194a499fee41c492e2294e5051fbe7"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "74a30d4e746076e097a79df6eb25c623"
server_key_iv: "5a15e0c9cf2f56ee8c0255a32b503a4e"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.635210 30010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:34.650390 30010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:34.653993 30010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:34.654227 30010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:34.654892 30010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:34.655802 30010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:34.655886 30010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:34.655975 30010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:34.656011 30010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:34.665998 30010 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:34471
I20260806 18:35:34.666003 30143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:34471 every 8 connection(s)
I20260806 18:35:34.666998 30010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:35:34.670691 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 30010
I20260806 18:35:34.670779 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:35:34.671077 18934 external_mini_cluster.cc:1428] Setting key 5e8927645e4a5ccabd8db7dcc10fec09
Aug 06 18:35:34 dist-test-slave-8w9v krb5kdc[29764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041334, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:35:34.684362 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.668967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:52359 (local address 127.18.125.190:37341)
0806 18:35:34.669116 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:34.669120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:34.670218 (+ 1098us) server_negotiation.cc:407] Connection header received
0806 18:35:34.671854 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:34.671857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:34.671919 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:34.672002 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:34.673512 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.674379 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.676480 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.676723 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.676819 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.680186 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:34.680195 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:34.680197 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:34.680229 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:34.681834 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:34.682513 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:34.682517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:34.682519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:34.682609 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:34.683002 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:34.683005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:34.683007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:34.683229 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:34.683349 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:34.684019 (+ 670us) server_negotiation.cc:299] Negotiation successful
0806 18:35:34.684214 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:35:34.685286 30146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.669497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:37341 (local address 127.18.125.130:52359)
0806 18:35:34.670061 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:34.670109 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:34.671636 (+ 1527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:34.672250 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:34.672260 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:34.672819 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:34.673340 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.673356 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.674567 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.674575 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.676323 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.676342 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.676922 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.676929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.677194 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.677981 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:34.678008 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:34.679905 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:34.682035 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:34.682044 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:34.682071 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:34.682379 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:34.682713 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:34.682717 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:34.682719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:34.682889 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:34.683349 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:34.683358 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:34.683669 (+ 311us) client_negotiation.cc:769] Sending connection context
0806 18:35:34.683942 (+ 273us) client_negotiation.cc:240] Negotiation successful
0806 18:35:34.684353 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
I20260806 18:35:34.687040 30144 heartbeater.cc:347] Connected to a master server at 127.18.125.190:37341
I20260806 18:35:34.687387 30144 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:34.687978 30144 heartbeater.cc:511] Master 127.18.125.190:37341 requested a full tablet report, sending...
I20260806 18:35:34.689375 29810 ts_manager.cc:194] Registered new tserver with Master: e9194a499fee41c492e2294e5051fbe7 (127.18.125.130:34471)
I20260806 18:35:34.690088 29810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:52359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:35:34.706939 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37341
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:35:34.834014 30149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:35:34.834411 30149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:35:34.834512 30149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:35:34.840451 30149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:35:34.840574 30149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:35:34.840682 30149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:35:34.846869 30149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:35:34.848083 30149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:35:34.849231 30149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:35:34.859903 30158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:34.860021 30149 server_base.cc:1034] running on GCE node
W20260806 18:35:34.859935 30155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:35:34.859905 30156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:35:34.860836 30149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:35:34.861594 30149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:35:34.862793 30149 hybrid_clock.cc:648] HybridClock initialized: now 1786041334862777 us; error 29 us; skew 500 ppm
Aug 06 18:35:34 dist-test-slave-8w9v krb5kdc[29764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041334, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:35:34.865826 30149 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:35:34.866762 30149 webserver.cc:460] Webserver started at http://127.18.125.131:38817/ using document root <none> and password file <none>
I20260806 18:35:34.867417 30149 fs_manager.cc:359] Metadata directory not provided
I20260806 18:35:34.867483 30149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:35:34.867728 30149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:35:34.870131 30149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "d1e994d922e844679d2fa86f80e8c2ba"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "afa04d6f3ceb20c7ddf38567b140c2a2"
server_key_iv: "c31cf2271f730bac156611cc914e9371"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.870749 30149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "d1e994d922e844679d2fa86f80e8c2ba"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "afa04d6f3ceb20c7ddf38567b140c2a2"
server_key_iv: "c31cf2271f730bac156611cc914e9371"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.874814 30149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:35:34.877708 30165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:34.878942 30149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:35:34.879063 30149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "d1e994d922e844679d2fa86f80e8c2ba"
format_stamp: "Formatted at 2026-08-06 18:35:34 on dist-test-slave-8w9v"
server_key: "afa04d6f3ceb20c7ddf38567b140c2a2"
server_key_iv: "c31cf2271f730bac156611cc914e9371"
server_key_version: "encryptionkey@0"
I20260806 18:35:34.879205 30149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:35:34.896277 30149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:35:34.900842 30149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:35:34.901089 30149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:35:34.902083 30149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:35:34.902992 30149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:35:34.903043 30149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:34.903085 30149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:35:34.903100 30149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:35:34.912807 30149 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:39851
I20260806 18:35:34.912931 30283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:39851 every 8 connection(s)
I20260806 18:35:34.913756 30149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:35:34.915104 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 30149
I20260806 18:35:34.915242 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:35:34.915618 18934 external_mini_cluster.cc:1428] Setting key 858a674516c10aedf7d9af4d9b6ae888
Aug 06 18:35:34 dist-test-slave-8w9v krb5kdc[29764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041334, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:35:34.929561 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.916057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:45651 (local address 127.18.125.190:37341)
0806 18:35:34.916277 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:34.916282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:34.917508 (+ 1226us) server_negotiation.cc:407] Connection header received
0806 18:35:34.919268 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:34.919271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:34.919331 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:34.919408 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:34.920732 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.921283 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.922519 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.922748 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.922859 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.925628 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:34.925631 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:34.925633 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:34.925660 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:34.927239 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:34.927811 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:34.927815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:34.927816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:34.927899 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:34.928340 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:34.928342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:34.928344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:34.928551 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:34.928629 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:34.929216 (+ 587us) server_negotiation.cc:299] Negotiation successful
0806 18:35:34.929376 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260806 18:35:34.930851 30286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.916636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:37341 (local address 127.18.125.131:45651)
0806 18:35:34.917312 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:34.917375 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:34.919046 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:34.919561 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:34.919568 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:34.919975 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:35:34.920594 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.920610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.921443 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.921446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.922379 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.922388 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.922912 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.922915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.923064 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.923776 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:35:34.923793 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:35:34.925453 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:35:34.927378 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:34.927384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:34.927397 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:34.927661 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:34.928003 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:35:34.928006 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:35:34.928007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:35:34.928223 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:35:34.928662 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:35:34.928670 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:35:34.928978 (+ 308us) client_negotiation.cc:769] Sending connection context
0806 18:35:34.929180 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 18:35:34.929561 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":151,"threads_started":1}
I20260806 18:35:34.932051 30284 heartbeater.cc:347] Connected to a master server at 127.18.125.190:37341
I20260806 18:35:34.932323 30284 heartbeater.cc:464] Registering TS with master...
I20260806 18:35:34.932893 30284 heartbeater.cc:511] Master 127.18.125.190:37341 requested a full tablet report, sending...
I20260806 18:35:34.933964 29810 ts_manager.cc:194] Registered new tserver with Master: d1e994d922e844679d2fa86f80e8c2ba (127.18.125.131:39851)
I20260806 18:35:34.934578 29810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:45651
I20260806 18:35:34.943986 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:35:34.954723 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.946672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44522 (local address 127.18.125.190:37341)
0806 18:35:34.946828 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:34.946833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:34.946851 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:35:34.946906 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:34.946911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:34.946966 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:34.947074 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:35:34.947849 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.948472 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.949360 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.949587 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.949709 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.951403 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:35:34.951407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:35:34.951409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:35:34.951439 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:35:34.952894 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:34.953353 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:34.953358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:34.953360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:34.953449 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:35:34.953738 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:35:34.953742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:35:34.953744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:35:34.953999 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:35:34.954150 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:34.954410 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 18:35:34.954544 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:35:34.958413 29810 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:35:34.961915 29810 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:35:34.978690 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.973008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60050 (local address 127.18.125.130:34471)
0806 18:35:34.973482 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:34.973517 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:34.973544 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:35:34.973632 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:34.973639 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:34.973896 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:34.974079 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:34.974489 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.975339 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.977115 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.977915 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.978068 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.978217 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:34.978399 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 18:35:34.978509 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":152,"threads_started":1}
I20260806 18:35:34.978684 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.972754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:34471 (local address 127.0.0.1:60050)
0806 18:35:34.973074 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:34.973089 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:34.973192 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:34.974096 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:34.974104 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:34.974133 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:34.974339 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.974347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.975491 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.975495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.976971 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.976982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.978053 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.978056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.978181 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.978221 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:35:34.978366 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 18:35:34.978466 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":68,"thread_start_us":112,"threads_started":1}
I20260806 18:35:34.978715 30295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.972208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:38535 (local address 127.0.0.1:37138)
0806 18:35:34.972756 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:34.972814 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:34.972947 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:34.974161 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:34.974164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:34.974177 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:34.974362 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.974371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.975874 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.975879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.977114 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.977121 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.978091 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.978095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.978198 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.978237 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:35:34.978313 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:35:34.978456 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":36,"thread_start_us":84,"threads_started":1}
I20260806 18:35:34.979312 30298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.972956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:39851 (local address 127.0.0.1:44250)
0806 18:35:34.973693 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:34.973707 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:34.973798 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:34.974454 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:34.974456 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:34.974475 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:34.974573 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.974578 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.976014 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.976017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.977555 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:34.977567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.978884 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.978888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:34.979000 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.979026 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:35:34.979098 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:35:34.979177 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260806 18:35:34.979677 30296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.972408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37138 (local address 127.18.125.129:38535)
0806 18:35:34.973683 (+ 1275us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:34.973690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:34.973717 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:35:34.973801 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:34.973806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:34.973989 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:34.974283 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:34.974503 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.975728 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.977247 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.977947 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.978849 (+ 902us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.979367 (+ 518us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:34.979426 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 18:35:34.979535 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1173,"thread_start_us":150,"threads_started":1}
I20260806 18:35:34.979797 30300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:34.973314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.18.125.131:39851)
0806 18:35:34.973942 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:34.973955 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:34.973981 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:35:34.974058 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:34.974063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:34.974244 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:34.974399 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:34.974693 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.975855 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.977710 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:34.978548 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:34.978671 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:34.978819 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:34.979523 (+ 704us) server_negotiation.cc:299] Negotiation successful
0806 18:35:34.979624 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":157,"threads_started":1}
I20260806 18:35:34.982484 29938 tablet_service.cc:1467] Processing CreateTablet for tablet 742ef984212b4863b56cfe42645f3609 (DEFAULT_TABLE table=test-table [id=4f8b835643a34e239f01fcdf2d0240d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:34.982489 30214 tablet_service.cc:1467] Processing CreateTablet for tablet 742ef984212b4863b56cfe42645f3609 (DEFAULT_TABLE table=test-table [id=4f8b835643a34e239f01fcdf2d0240d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:34.982494 30075 tablet_service.cc:1467] Processing CreateTablet for tablet 742ef984212b4863b56cfe42645f3609 (DEFAULT_TABLE table=test-table [id=4f8b835643a34e239f01fcdf2d0240d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:35:34.983632 29938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 742ef984212b4863b56cfe42645f3609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:34.983635 30075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 742ef984212b4863b56cfe42645f3609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:34.983644 30214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 742ef984212b4863b56cfe42645f3609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:35:34.990865 30301 tablet_bootstrap.cc:492] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006: Bootstrap starting.
I20260806 18:35:34.992262 30303 tablet_bootstrap.cc:492] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba: Bootstrap starting.
I20260806 18:35:34.992484 30302 tablet_bootstrap.cc:492] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7: Bootstrap starting.
I20260806 18:35:34.993415 30301 tablet_bootstrap.cc:654] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:34.994206 30301 log.cc:824] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:34.994521 30303 tablet_bootstrap.cc:654] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:34.995038 30302 tablet_bootstrap.cc:654] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7: Neither blocks nor log segments found. Creating new log.
I20260806 18:35:34.995869 30302 log.cc:824] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:34.995869 30303 log.cc:824] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba: Log is configured to *not* fsync() on all Append() calls
I20260806 18:35:34.996613 30301 tablet_bootstrap.cc:492] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006: No bootstrap required, opened a new log
I20260806 18:35:34.996845 30301 ts_tablet_manager.cc:1397] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:35:34.998435 30302 tablet_bootstrap.cc:492] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7: No bootstrap required, opened a new log
I20260806 18:35:34.998682 30302 ts_tablet_manager.cc:1397] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:35:34.998802 30303 tablet_bootstrap.cc:492] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba: No bootstrap required, opened a new log
I20260806 18:35:34.998986 30303 ts_tablet_manager.cc:1397] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:35:35.000874 30301 raft_consensus.cc:348] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607d44838e334715b56165a350a44006" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38535 } } peers { permanent_uuid: "d1e994d922e844679d2fa86f80e8c2ba" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39851 } } peers { permanent_uuid: "e9194a499fee41c492e2294e5051fbe7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34471 } }
I20260806 18:35:35.001133 30301 raft_consensus.cc:374] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:35.001186 30301 raft_consensus.cc:729] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607d44838e334715b56165a350a44006, State: Initialized, Role: FOLLOWER
I20260806 18:35:35.001643 30301 consensus_queue.cc:260] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607d44838e334715b56165a350a44006" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38535 } } peers { permanent_uuid: "d1e994d922e844679d2fa86f80e8c2ba" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39851 } } peers { permanent_uuid: "e9194a499fee41c492e2294e5051fbe7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34471 } }
I20260806 18:35:35.002398 30005 heartbeater.cc:503] Master 127.18.125.190:37341 was elected leader, sending a full tablet report...
I20260806 18:35:35.002775 30301 ts_tablet_manager.cc:1428] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:35:35.002784 30302 raft_consensus.cc:348] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607d44838e334715b56165a350a44006" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38535 } } peers { permanent_uuid: "d1e994d922e844679d2fa86f80e8c2ba" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39851 } } peers { permanent_uuid: "e9194a499fee41c492e2294e5051fbe7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34471 } }
I20260806 18:35:35.003029 30302 raft_consensus.cc:374] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:35.003089 30302 raft_consensus.cc:729] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9194a499fee41c492e2294e5051fbe7, State: Initialized, Role: FOLLOWER
I20260806 18:35:35.003523 30302 consensus_queue.cc:260] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607d44838e334715b56165a350a44006" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38535 } } peers { permanent_uuid: "d1e994d922e844679d2fa86f80e8c2ba" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39851 } } peers { permanent_uuid: "e9194a499fee41c492e2294e5051fbe7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34471 } }
I20260806 18:35:35.004184 30303 raft_consensus.cc:348] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607d44838e334715b56165a350a44006" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38535 } } peers { permanent_uuid: "d1e994d922e844679d2fa86f80e8c2ba" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39851 } } peers { permanent_uuid: "e9194a499fee41c492e2294e5051fbe7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34471 } }
I20260806 18:35:35.004436 30303 raft_consensus.cc:374] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:35:35.004487 30303 raft_consensus.cc:729] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1e994d922e844679d2fa86f80e8c2ba, State: Initialized, Role: FOLLOWER
I20260806 18:35:35.004583 30302 ts_tablet_manager.cc:1428] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:35:35.004626 30144 heartbeater.cc:503] Master 127.18.125.190:37341 was elected leader, sending a full tablet report...
I20260806 18:35:35.004889 30303 consensus_queue.cc:260] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607d44838e334715b56165a350a44006" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38535 } } peers { permanent_uuid: "d1e994d922e844679d2fa86f80e8c2ba" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39851 } } peers { permanent_uuid: "e9194a499fee41c492e2294e5051fbe7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34471 } }
I20260806 18:35:35.005492 30284 heartbeater.cc:503] Master 127.18.125.190:37341 was elected leader, sending a full tablet report...
I20260806 18:35:35.005811 30303 ts_tablet_manager.cc:1428] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260806 18:35:35.030333 30308 raft_consensus.cc:482] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:35:35.030576 30308 raft_consensus.cc:504] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607d44838e334715b56165a350a44006" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38535 } } peers { permanent_uuid: "d1e994d922e844679d2fa86f80e8c2ba" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39851 } } peers { permanent_uuid: "e9194a499fee41c492e2294e5051fbe7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34471 } }
I20260806 18:35:35.031831 30308 leader_election.cc:288] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1e994d922e844679d2fa86f80e8c2ba (127.18.125.131:39851), e9194a499fee41c492e2294e5051fbe7 (127.18.125.130:34471)
I20260806 18:35:35.037812 30300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:35.032378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:41367 (local address 127.18.125.131:39851)
0806 18:35:35.032558 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:35.032565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:35.032992 (+ 427us) server_negotiation.cc:407] Connection header received
0806 18:35:35.033055 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:35.033058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:35.033116 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:35.033223 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:35.033642 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:35.034514 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:35.036446 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:35.037001 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:35.037086 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:35.037162 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:35.037573 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 18:35:35.037686 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 18:35:35.037825 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:35.032389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:48883 (local address 127.18.125.130:34471)
0806 18:35:35.032593 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:35.032602 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:35.032857 (+ 255us) server_negotiation.cc:407] Connection header received
0806 18:35:35.033053 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:35.033057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:35.033118 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:35.033223 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:35:35.033642 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:35.034516 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:35.036493 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:35.037267 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:35.037397 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:35.037435 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:35.037644 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 18:35:35.037720 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260806 18:35:35.037825 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:35.032215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:34471 (local address 127.18.125.129:48883)
0806 18:35:35.032741 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:35.032758 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:35.032918 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:35.033250 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:35.033253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:35.033294 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:35.033509 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:35.033516 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:35.034662 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:35.034667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:35.036280 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:35.036293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:35.037394 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:35.037397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:35.037518 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:35.037542 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:35:35.037611 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:35:35.037661 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":168,"spinlock_wait_cycles":39936,"thread_start_us":167,"threads_started":1}
I20260806 18:35:35.037840 30311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:35.032215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:39851 (local address 127.18.125.129:41367)
0806 18:35:35.032853 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:35.032868 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:35:35.032956 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:35:35.033250 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:35:35.033253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:35:35.033295 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:35:35.033509 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:35.033516 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:35.034662 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:35.034667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:35.036280 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:35:35.036293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:35:35.037268 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:35:35.037271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:35:35.037387 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:35.037411 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:35:35.037527 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 18:35:35.037626 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":280,"spinlock_wait_cycles":7296,"thread_start_us":135,"threads_started":1}
I20260806 18:35:35.038565 30096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "742ef984212b4863b56cfe42645f3609" candidate_uuid: "607d44838e334715b56165a350a44006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9194a499fee41c492e2294e5051fbe7" is_pre_election: true
I20260806 18:35:35.039005 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "742ef984212b4863b56cfe42645f3609" candidate_uuid: "607d44838e334715b56165a350a44006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1e994d922e844679d2fa86f80e8c2ba" is_pre_election: true
I20260806 18:35:35.039134 30096 raft_consensus.cc:2454] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 607d44838e334715b56165a350a44006 in term 0.
I20260806 18:35:35.039325 30235 raft_consensus.cc:2454] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 607d44838e334715b56165a350a44006 in term 0.
I20260806 18:35:35.039860 29892 leader_election.cc:302] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 607d44838e334715b56165a350a44006, d1e994d922e844679d2fa86f80e8c2ba, e9194a499fee41c492e2294e5051fbe7; no voters:
I20260806 18:35:35.040210 30308 raft_consensus.cc:2781] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:35:35.040355 30308 raft_consensus.cc:482] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:35:35.040424 30308 raft_consensus.cc:3037] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:35.041574 30308 raft_consensus.cc:504] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607d44838e334715b56165a350a44006" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38535 } } peers { permanent_uuid: "d1e994d922e844679d2fa86f80e8c2ba" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39851 } } peers { permanent_uuid: "e9194a499fee41c492e2294e5051fbe7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34471 } }
I20260806 18:35:35.041958 30308 leader_election.cc:288] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [CANDIDATE]: Term 1 election: Requested vote from peers d1e994d922e844679d2fa86f80e8c2ba (127.18.125.131:39851), e9194a499fee41c492e2294e5051fbe7 (127.18.125.130:34471)
I20260806 18:35:35.042346 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "742ef984212b4863b56cfe42645f3609" candidate_uuid: "607d44838e334715b56165a350a44006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1e994d922e844679d2fa86f80e8c2ba"
I20260806 18:35:35.042500 30235 raft_consensus.cc:3037] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:35.042510 30096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "742ef984212b4863b56cfe42645f3609" candidate_uuid: "607d44838e334715b56165a350a44006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9194a499fee41c492e2294e5051fbe7"
I20260806 18:35:35.042654 30096 raft_consensus.cc:3037] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:35:35.043764 30235 raft_consensus.cc:2454] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607d44838e334715b56165a350a44006 in term 1.
I20260806 18:35:35.043780 30096 raft_consensus.cc:2454] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607d44838e334715b56165a350a44006 in term 1.
I20260806 18:35:35.044085 29892 leader_election.cc:302] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 607d44838e334715b56165a350a44006, d1e994d922e844679d2fa86f80e8c2ba; no voters:
I20260806 18:35:35.044320 30308 raft_consensus.cc:2781] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:35:35.044607 30308 raft_consensus.cc:686] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 1 LEADER]: Becoming Leader. State: Replica: 607d44838e334715b56165a350a44006, State: Running, Role: LEADER
I20260806 18:35:35.044998 30308 consensus_queue.cc:237] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607d44838e334715b56165a350a44006" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38535 } } peers { permanent_uuid: "d1e994d922e844679d2fa86f80e8c2ba" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39851 } } peers { permanent_uuid: "e9194a499fee41c492e2294e5051fbe7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34471 } }
I20260806 18:35:35.048599 29810 catalog_manager.cc:5495] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 reported cstate change: term changed from 0 to 1, leader changed from <none> to 607d44838e334715b56165a350a44006 (127.18.125.129). New cstate: current_term: 1 leader_uuid: "607d44838e334715b56165a350a44006" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "607d44838e334715b56165a350a44006" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1e994d922e844679d2fa86f80e8c2ba" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9194a499fee41c492e2294e5051fbe7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34471 } health_report { overall_health: UNKNOWN } } }
I20260806 18:35:35.062372 30296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:35.057403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37140 (local address 127.18.125.129:38535)
0806 18:35:35.057547 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:35:35.057551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:35:35.057567 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:35:35.057721 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:35:35.057723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:35:35.057782 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:35:35.057875 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:35:35.058212 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:35.058757 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:35.060031 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:35:35.060406 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:35:35.060527 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:35:35.060893 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:35:35.061744 (+ 851us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:35:35.061927 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 18:35:35.062076 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 18:35:35.062160 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:35:35.069157 30096 raft_consensus.cc:1261] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7 [term 1 FOLLOWER]: Refusing update from remote peer 607d44838e334715b56165a350a44006: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:35.069157 30235 raft_consensus.cc:1261] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba [term 1 FOLLOWER]: Refusing update from remote peer 607d44838e334715b56165a350a44006: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:35:35.070062 30313 consensus_queue.cc:1035] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1e994d922e844679d2fa86f80e8c2ba" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 39851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:35.070303 30308 consensus_queue.cc:1035] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9194a499fee41c492e2294e5051fbe7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:35:35.082273 30317 mvcc.cc:205] Tried to move back new op lower bound from 7315625308434386944 to 7315625308347580416. Current Snapshot: MvccSnapshot[applied={T|T < 7315625308434386944}]
I20260806 18:35:35.088354 29810 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:35.088541 29810 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:35:35.091493 29810 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1153529a57fe4f4e8a880db64f3632f0: Sending DeleteTablet for 3 replicas of tablet 742ef984212b4863b56cfe42645f3609
I20260806 18:35:35.092595 29938 tablet_service.cc:1514] Processing DeleteTablet for tablet 742ef984212b4863b56cfe42645f3609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:35 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:37138
I20260806 18:35:35.092600 30075 tablet_service.cc:1514] Processing DeleteTablet for tablet 742ef984212b4863b56cfe42645f3609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:35 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:60050
I20260806 18:35:35.093147 30325 tablet_replica.cc:330] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006: stopping tablet replica
I20260806 18:35:35.093194 30214 tablet_service.cc:1514] Processing DeleteTablet for tablet 742ef984212b4863b56cfe42645f3609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:35:35 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:44250
I20260806 18:35:35.093549 30325 raft_consensus.cc:2229] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:35:35.093711 30327 tablet_replica.cc:330] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba: stopping tablet replica
I20260806 18:35:35.094004 30327 raft_consensus.cc:2229] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:35.094053 30325 raft_consensus.cc:2258] T 742ef984212b4863b56cfe42645f3609 P 607d44838e334715b56165a350a44006 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:35.094225 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 29875
I20260806 18:35:35.094342 30327 raft_consensus.cc:2258] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:35.096971 30326 tablet_replica.cc:330] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7: stopping tablet replica
I20260806 18:35:35.097308 30326 raft_consensus.cc:2229] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:35:35.097481 30327 ts_tablet_manager.cc:1905] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:35.097662 30326 raft_consensus.cc:2258] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:35:35.098951 30326 ts_tablet_manager.cc:1905] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:35:35.101321 30326 ts_tablet_manager.cc:1918] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:35.101459 30326 log.cc:1196] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/742ef984212b4863b56cfe42645f3609
I20260806 18:35:35.101590 30327 ts_tablet_manager.cc:1918] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:35:35.101719 30327 log.cc:1196] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/742ef984212b4863b56cfe42645f3609
I20260806 18:35:35.101902 30326 ts_tablet_manager.cc:1939] T 742ef984212b4863b56cfe42645f3609 P e9194a499fee41c492e2294e5051fbe7: Deleting consensus metadata
I20260806 18:35:35.102114 30327 ts_tablet_manager.cc:1939] T 742ef984212b4863b56cfe42645f3609 P d1e994d922e844679d2fa86f80e8c2ba: Deleting consensus metadata
I20260806 18:35:35.103304 29797 catalog_manager.cc:4841] TS e9194a499fee41c492e2294e5051fbe7 (127.18.125.130:34471): tablet 742ef984212b4863b56cfe42645f3609 (table test-table [id=4f8b835643a34e239f01fcdf2d0240d6]) successfully deleted
I20260806 18:35:35.103499 29795 catalog_manager.cc:4841] TS d1e994d922e844679d2fa86f80e8c2ba (127.18.125.131:39851): tablet 742ef984212b4863b56cfe42645f3609 (table test-table [id=4f8b835643a34e239f01fcdf2d0240d6]) successfully deleted
W20260806 18:35:35.104662 29795 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:38535 (error 108)
I20260806 18:35:35.105526 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 30010
I20260806 18:35:35.105773 30298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:35:35.105463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:38535 (local address 127.0.0.1:37156)
0806 18:35:35.105600 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:35:35.105676 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:38535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 18:35:35.108438 29795 catalog_manager.cc:4584] TS 607d44838e334715b56165a350a44006 (127.18.125.129:38535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 742ef984212b4863b56cfe42645f3609: Network error: Client connection negotiation failed: client connection to 127.18.125.129:38535: connect: Connection refused (error 111)
I20260806 18:35:35.114096 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 30149
I20260806 18:35:35.122206 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 29777
2026-08-06T18:35:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2410 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:35:35 dist-test-slave-8w9v krb5kdc[30331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:35:35 dist-test-slave-8w9v krb5kdc[30331](info): set up 2 sockets
Aug 06 18:35:35 dist-test-slave-8w9v krb5kdc[30331](info): commencing operation
krb5kdc: starting...
W20260806 18:35:36.211519 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:35:36 dist-test-slave-8w9v krb5kdc[30331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:35:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:35:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/rangeradmin_127.18.125.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/rangeradmin_127.18.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/rangerlookup_127.18.125.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/rangerlookup_127.18.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/HTTP_127.18.125.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/HTTP_127.18.125.148@KRBTEST.COM.keytab.
I20260806 18:35:36.404457 18934 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskun4LBX/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/postgres -l logfile start
2026-08-06 18:35:39.397 UTC [30363] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 18:35:39.397 UTC [30363] LOG: listening on IPv4 address "127.18.125.148", port 33441
2026-08-06 18:35:39.402 UTC [30363] LOG: listening on Unix socket "/tmp/.s.PGSQL.33441"
2026-08-06 18:35:39.439 UTC [30366] LOG: database system was shut down at 2026-08-06 18:35:37 UTC
2026-08-06 18:35:39.444 UTC [30363] LOG: database system is ready to accept connections
I20260806 18:35:40.450491 18934 mini_postgres.cc:96] Postgres bound to 33441
2026-08-06 18:35:40.458 UTC [30375] FATAL: database "slave" does not exist
127.18.125.148:33441 - accepting connections
I20260806 18:35:40.458873 18934 mini_ranger.cc:161] Starting Ranger...
I20260806 18:35:40.475016 18934 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 18:35:41.072856 18934 mini_ranger.cc:87] Created ranger Postgres database
I20260806 18:35:41.072940 18934 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-admin
2026-08-06 18:35:41,423 [I] DB FLAVOR :POSTGRES
2026-08-06 18:35:41,424 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:35:41,424 [I] Checking connection..
2026-08-06 18:35:41,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 18:35:41,730 [I] Checking connection passed.
2026-08-06 18:35:41,730 [I] --------- Verifying version history table ---------
2026-08-06 18:35:41,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:35:42,007 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:35:42,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:35:42,283 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:35:42,283 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 18:35:42,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 18:35:42.532 UTC [30443] WARNING: there is no transaction in progress
2026-08-06 18:35:42,540 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 18:35:42,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:35:42,826 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 18:35:42,827 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:35:42,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 18:35:43,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 18:35:43,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8w9v.c.gcp-upstream.internal','N') ;"
2026-08-06 18:35:43,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:35:43,947 [I] Table x_portal_user does not exist in database ranger
2026-08-06 18:35:43,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:35:44,234 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 18:35:44,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:35:44,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:35:44,785 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 18:35:44,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 18:35:45.938 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:45.963 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:45.986 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.007 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.025 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.047 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.111 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.119 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.135 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.147 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.160 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.177 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.188 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.196 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.205 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:46.215 UTC [30556] WARNING: there is no transaction in progress
2026-08-06 18:35:47,045 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 18:35:47,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 18:35:47,300 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 18:35:47,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:35:47,591 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 18:35:47,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:35:47,875 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 18:35:47,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:35:48,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:35:48,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8w9v.c.gcp-upstream.internal';"
2026-08-06 18:35:48,672 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 18:35:48,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 18:35:48,939 [I] DB_PATCHES have already been applied
I20260806 18:35:48.947060 18934 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-admin:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskun4LBX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 18:35:48.947166 18934 mini_ranger.cc:193] Using host: 127.18.125.148
I20260806 18:35:48.952199 18934 mini_ranger.cc:227] Ranger admin URL: http://127.18.125.148:56899
Aug 06, 2026 6:35:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:35:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:35:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:35:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:35:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:35:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:35:50 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 18:35:50.107379 18934 mini_ranger.cc:160] Time spent starting Ranger: real 9.649s user 0.002s sys 0.011s
Aug 06, 2026 6:36:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:36:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:36:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:36:13 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:36:14 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 18:36:15.655000 18934 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/rangerkms_127.18.125.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/rangerkms_127.18.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.125.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.125.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.125.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/HTTP_127.18.125.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.125.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/HTTP_127.18.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 18:36:15 dist-test-slave-8w9v krb5kdc[30331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041375, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 18:36:15.765986 18934 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 18:36:15.778673 18934 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 18:36:16.836330 18934 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 18:36:16.836449 18934 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-kms
I20260806 18:36:16.836592 18934 mini_ranger_kms.cc:227] Using postgres at 127.18.125.148:33441
2026-08-06 18:36:17,149 [I] DB FLAVOR :POSTGRES
2026-08-06 18:36:17,149 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:36:17,149 [I] Checking connection
2026-08-06 18:36:17,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 18:36:17,439 [I] connection success
2026-08-06 18:36:17,440 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 18:36:17,440 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 18:36:17,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 18:36:17,728 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 18:36:17,729 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:36:17,729 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 18:36:17,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:33441/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 18:36:18,061 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 18:36:18.606607 18934 mini_ranger_kms.cc:310] Created kms service
I20260806 18:36:18.820086 18934 mini_ranger_kms.cc:326] Created kudu user
I20260806 18:36:18.880422 18934 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 18:36:18 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041375, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:36:19.406454 18934 mini_ranger_kms.cc:384] Added ranger policy
I20260806 18:36:19.406622 18934 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-kms:/tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 18:36:19.406644 18934 mini_ranger_kms.cc:240] Using host: 127.18.125.148
I20260806 18:36:19.412027 18934 mini_ranger_kms.cc:276] Ranger KMS PID: 30742
I20260806 18:36:19.412215 18934 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.125.148:38747
Aug 06, 2026 6:36:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:36:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:36:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:36:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:36:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:36:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 18:36:21.771202 18934 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.005s user 0.011s sys 0.283s
I20260806 18:36:21.771363 18934 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 18:36:21.771528 18934 mini_ranger_kms.cc:401] 127.18.125.148:38747/kms/v1/keys
Aug 06 18:36:21 dist-test-slave-8w9v krb5kdc[30331](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041381, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.125.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 18:36:22 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041381, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.125.148@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
Aug 06 18:36:22 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041381, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.125.148@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
Aug 06 18:36:25 dist-test-slave-8w9v krb5kdc[30331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:36:25.189743 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:44167
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:45617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:44167
--ranger_config_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:38747/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:25.317692 30794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:25.318022 30794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:25.318065 30794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:25.322458 30794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:36:25.322547 30794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:25.322568 30794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:25.322584 30794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:36:25.322600 30794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:36:25.328974 30794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:45617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.125.148:38747/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:44167
--ranger_config_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:44167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:36:25.330245 30794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:25.331403 30794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:25.342700 30800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:25.342684 30799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:25.342876 30794 server_base.cc:1034] running on GCE node
W20260806 18:36:25.342684 30802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:25.343813 30794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:25.345307 30794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:25.346503 30794 hybrid_clock.cc:648] HybridClock initialized: now 1786041385346478 us; error 56 us; skew 500 ppm
Aug 06 18:36:25 dist-test-slave-8w9v krb5kdc[30331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041385, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:25.349648 30794 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:36:25.350826 30794 webserver.cc:460] Webserver started at http://127.18.125.190:42987/ using document root <none> and password file <none>
I20260806 18:36:25.351456 30794 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:25.351517 30794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:25.351760 30794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:36:25 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041385, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:36:25.406360 30794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "c782ab79490e4df2a581a69f3fab230b"
format_stamp: "Formatted at 2026-08-06 18:36:25 on dist-test-slave-8w9v"
server_key: "f5af2f2b09582a6fccb89243da927b8f"
server_key_iv: "a3b9517e4a02880114b1f6f2fb22b2eb"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:25.407188 30794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "c782ab79490e4df2a581a69f3fab230b"
format_stamp: "Formatted at 2026-08-06 18:36:25 on dist-test-slave-8w9v"
server_key: "f5af2f2b09582a6fccb89243da927b8f"
server_key_iv: "a3b9517e4a02880114b1f6f2fb22b2eb"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:25.411708 30794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260806 18:36:25.428403 30809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:25.430060 30794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:36:25.430258 30794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "c782ab79490e4df2a581a69f3fab230b"
format_stamp: "Formatted at 2026-08-06 18:36:25 on dist-test-slave-8w9v"
server_key: "f5af2f2b09582a6fccb89243da927b8f"
server_key_iv: "a3b9517e4a02880114b1f6f2fb22b2eb"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:25.430425 30794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:25.455269 30794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:25.458843 30794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:25.459081 30794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:25.470372 30794 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:44167
I20260806 18:36:25.470418 30864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:44167 every 8 connection(s)
I20260806 18:36:25.471510 30794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:36:25.474985 30866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:25.479398 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 30794
I20260806 18:36:25.479540 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:36:25.479876 18934 external_mini_cluster.cc:1428] Setting key df85050123720045e692b869f0b851a5
I20260806 18:36:25.482431 30866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b: Bootstrap starting.
I20260806 18:36:25.485610 30866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:25.486589 30866 log.cc:824] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:25.489002 30866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b: No bootstrap required, opened a new log
Aug 06 18:36:25 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:36:25.494117 30866 raft_consensus.cc:348] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c782ab79490e4df2a581a69f3fab230b" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44167 } }
I20260806 18:36:25.494480 30866 raft_consensus.cc:374] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:25.494544 30866 raft_consensus.cc:729] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c782ab79490e4df2a581a69f3fab230b, State: Initialized, Role: FOLLOWER
I20260806 18:36:25.495119 30866 consensus_queue.cc:260] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c782ab79490e4df2a581a69f3fab230b" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44167 } }
I20260806 18:36:25.495311 30866 raft_consensus.cc:388] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:36:25.495404 30866 raft_consensus.cc:482] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:36:25.495527 30866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:25.496975 30866 raft_consensus.cc:504] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c782ab79490e4df2a581a69f3fab230b" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44167 } }
I20260806 18:36:25.497398 30866 leader_election.cc:302] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c782ab79490e4df2a581a69f3fab230b; no voters:
I20260806 18:36:25.497771 30866 leader_election.cc:288] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:36:25.497926 30871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:36:25.498159 30871 raft_consensus.cc:686] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [term 1 LEADER]: Becoming Leader. State: Replica: c782ab79490e4df2a581a69f3fab230b, State: Running, Role: LEADER
I20260806 18:36:25.498498 30871 consensus_queue.cc:237] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c782ab79490e4df2a581a69f3fab230b" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44167 } }
I20260806 18:36:25.498586 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:25.481638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44636 (local address 127.18.125.190:44167)
0806 18:36:25.482159 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:25.482180 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:25.482227 (+ 47us) server_negotiation.cc:407] Connection header received
0806 18:36:25.483133 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:25.483159 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:25.483616 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:25.484090 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:25.485189 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:25.486766 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:25.488001 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:25.488714 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:25.488893 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:25.492271 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:25.492279 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:25.492300 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:25.492340 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:25.494519 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:25.495169 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:25.495175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:25.495182 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:25.495333 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:25.495677 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:25.495684 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:25.495688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:25.496302 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:25.496518 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:25.496837 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 18:36:25.497284 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":160,"threads_started":1}
I20260806 18:36:25.499256 30866 sys_catalog.cc:564] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:36:25.499890 30871 sys_catalog.cc:455] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c782ab79490e4df2a581a69f3fab230b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c782ab79490e4df2a581a69f3fab230b" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44167 } } }
I20260806 18:36:25.500005 30871 sys_catalog.cc:458] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:25.500248 30871 sys_catalog.cc:455] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c782ab79490e4df2a581a69f3fab230b. Latest consensus state: current_term: 1 leader_uuid: "c782ab79490e4df2a581a69f3fab230b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c782ab79490e4df2a581a69f3fab230b" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44167 } } }
I20260806 18:36:25.500324 30871 sys_catalog.cc:458] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b [sys.catalog]: This master's current role is: LEADER
I20260806 18:36:25.501878 30866 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:36:25.506896 30877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:36:25.511853 30877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:36:25.521811 30877 catalog_manager.cc:1347] Generated new cluster ID: 6cbe78d4bd6147e09e088503d552597f
I20260806 18:36:25.522038 30877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:36:25.544759 30877 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:36:25.546625 30877 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:36:25.558950 30877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b: Generated new TSK 0
I20260806 18:36:25.560173 30877 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:36:26.375808 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:38747/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44167
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:45617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:36:26.515444 30913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:26.515762 30913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:26.515842 30913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:26.520211 30913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:26.520308 30913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:26.520404 30913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:36:26.526633 30913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:45617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.125.148:38747/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:36:26.527913 30913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:26.529377 30913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:26.541597 30921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:26.541564 30919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:26.541580 30918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:26.541997 30913 server_base.cc:1034] running on GCE node
I20260806 18:36:26.542497 30913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:26.543365 30913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:26.544621 30913 hybrid_clock.cc:648] HybridClock initialized: now 1786041386544584 us; error 52 us; skew 500 ppm
Aug 06 18:36:26 dist-test-slave-8w9v krb5kdc[30331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041386, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:26.548512 30913 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:36:26.549774 30913 webserver.cc:460] Webserver started at http://127.18.125.129:42521/ using document root <none> and password file <none>
I20260806 18:36:26.550542 30913 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:26.550637 30913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:26.550899 30913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:36:26 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041386, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:36:26.574827 30913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "3ddd5d279a83477db94942d3c3326ca6"
format_stamp: "Formatted at 2026-08-06 18:36:26 on dist-test-slave-8w9v"
server_key: "47136cd00ebbce8977d2d72d020b4674"
server_key_iv: "951bb9efb288b60c96eaefabca0f2ce4"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:26.575904 30913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "3ddd5d279a83477db94942d3c3326ca6"
format_stamp: "Formatted at 2026-08-06 18:36:26 on dist-test-slave-8w9v"
server_key: "47136cd00ebbce8977d2d72d020b4674"
server_key_iv: "951bb9efb288b60c96eaefabca0f2ce4"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:26.581033 30913 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:36:26.600394 30930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:26.602639 30913 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 18:36:26.602829 30913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "3ddd5d279a83477db94942d3c3326ca6"
format_stamp: "Formatted at 2026-08-06 18:36:26 on dist-test-slave-8w9v"
server_key: "47136cd00ebbce8977d2d72d020b4674"
server_key_iv: "951bb9efb288b60c96eaefabca0f2ce4"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:26.603001 30913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:26.629786 30913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:26.634320 30913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:26.634568 30913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:26.635540 30913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:26.636891 30913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:26.636970 30913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:26.637035 30913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:26.637058 30913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:26.650086 30913 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:37789
I20260806 18:36:26.651510 30913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:36:26.652688 31048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:37789 every 8 connection(s)
I20260806 18:36:26.656788 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 30913
I20260806 18:36:26.656946 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:36:26.657279 18934 external_mini_cluster.cc:1428] Setting key 6d3946fa2491e4a35df8fd0728216c5e
Aug 06 18:36:26 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041386, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:36:26.675246 31051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:26.653989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:48391 (local address 127.18.125.190:44167)
0806 18:36:26.654323 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:26.654328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:26.655185 (+ 857us) server_negotiation.cc:407] Connection header received
0806 18:36:26.656463 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:26.656467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:26.656535 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:26.656644 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:26.662808 (+ 6164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:26.663702 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:26.665681 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:26.666073 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:26.666163 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:26.670159 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:26.670164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:26.670167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:26.670202 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:26.672355 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:26.673100 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:26.673104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:26.673106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:26.673208 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:26.673678 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:26.673682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:26.673684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:26.673950 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:26.674041 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:26.674812 (+ 771us) server_negotiation.cc:299] Negotiation successful
0806 18:36:26.675046 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":109,"threads_started":1}
I20260806 18:36:26.677408 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:26.654413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44167 (local address 127.18.125.129:48391)
0806 18:36:26.654971 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:26.655031 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:26.656175 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:26.658279 (+ 2104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:26.658291 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:26.658692 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:26.660952 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:26.660980 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:26.663903 (+ 2923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:26.663906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:26.665507 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:26.665520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:26.666436 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:26.666441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:26.666675 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:26.667951 (+ 1276us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:26.667974 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:26.669888 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:26.672526 (+ 2638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:26.672533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:26.672552 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:26.672961 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:26.673331 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:26.673334 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:26.673336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:26.673552 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:26.674117 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:26.674125 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:26.674487 (+ 362us) client_negotiation.cc:769] Sending connection context
0806 18:36:26.675784 (+ 1297us) client_negotiation.cc:240] Negotiation successful
0806 18:36:26.676256 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":152,"threads_started":1}
I20260806 18:36:26.679418 31049 heartbeater.cc:347] Connected to a master server at 127.18.125.190:44167
I20260806 18:36:26.679718 31049 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:26.680413 31049 heartbeater.cc:511] Master 127.18.125.190:44167 requested a full tablet report, sending...
I20260806 18:36:26.682624 30819 ts_manager.cc:194] Registered new tserver with Master: 3ddd5d279a83477db94942d3c3326ca6 (127.18.125.129:37789)
I20260806 18:36:26.684319 30819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:48391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:36:26.714480 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:38747/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44167
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:45617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:36:26 dist-test-slave-8w9v krb5kdc[30331](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041386, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 18:36:26.895445 31057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:26.895885 31057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:26.895987 31057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:26.902829 31057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:26.902940 31057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:26.903036 31057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:36:26.912022 31057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:45617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.125.148:38747/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:36:26.913692 31057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:26.915381 31057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:26.929603 31064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:26.929739 31063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:26.929924 31066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:26.930063 31057 server_base.cc:1034] running on GCE node
I20260806 18:36:26.930615 31057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:26.931569 31057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:26.934887 31057 hybrid_clock.cc:648] HybridClock initialized: now 1786041386934769 us; error 113 us; skew 500 ppm
Aug 06 18:36:26 dist-test-slave-8w9v krb5kdc[30331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041386, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:26.939481 31057 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:36:26.941052 31057 webserver.cc:460] Webserver started at http://127.18.125.130:46779/ using document root <none> and password file <none>
I20260806 18:36:26.942021 31057 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:26.942272 31057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:26.942692 31057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:36:26 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041386, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:36:26.963817 31057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "93fa408d88bb4ea9a091ebe01f8e9693"
format_stamp: "Formatted at 2026-08-06 18:36:26 on dist-test-slave-8w9v"
server_key: "04e39eb28c82977305357a0350d38c86"
server_key_iv: "dd016a6dcf52f120e884678f50326599"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:26.964973 31057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "93fa408d88bb4ea9a091ebe01f8e9693"
format_stamp: "Formatted at 2026-08-06 18:36:26 on dist-test-slave-8w9v"
server_key: "04e39eb28c82977305357a0350d38c86"
server_key_iv: "dd016a6dcf52f120e884678f50326599"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:26.978055 31057 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.004s sys 0.000s
I20260806 18:36:27.000012 31073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:27.001906 31057 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 18:36:27.002156 31057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "93fa408d88bb4ea9a091ebe01f8e9693"
format_stamp: "Formatted at 2026-08-06 18:36:26 on dist-test-slave-8w9v"
server_key: "04e39eb28c82977305357a0350d38c86"
server_key_iv: "dd016a6dcf52f120e884678f50326599"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:27.002442 31057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:27.019193 31057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:27.023892 31057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:27.024245 31057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:27.025085 31057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:27.026412 31057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:27.026530 31057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:27.026641 31057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:27.026719 31057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:27.043052 31057 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:34333
I20260806 18:36:27.044502 31057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:36:27.046578 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 31057
I20260806 18:36:27.046715 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:36:27.047062 18934 external_mini_cluster.cc:1428] Setting key 2ec9b498a6a8bd592f1f50297af9a6ac
Aug 06 18:36:27 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041386, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:36:27.046916 31191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:34333 every 8 connection(s)
I20260806 18:36:27.063054 31051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.050524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:53545 (local address 127.18.125.190:44167)
0806 18:36:27.050676 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:27.050683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:27.050701 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:36:27.051179 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:27.051182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:27.051238 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:27.051320 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:27.052561 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.053110 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.054290 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.054529 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.054648 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.057960 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:27.057966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:27.057969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:27.058002 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:27.059838 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:27.060619 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:27.060623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:27.060625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:27.060731 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:27.061411 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:27.061414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:27.061416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:27.061674 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:27.061768 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:27.062716 (+ 948us) server_negotiation.cc:299] Negotiation successful
0806 18:36:27.062876 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:36:27.065559 31194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.049267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44167 (local address 127.18.125.130:53545)
0806 18:36:27.049901 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:27.049954 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:27.050948 (+ 994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:27.051496 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:27.051503 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:27.051796 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:27.052408 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.052423 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.053253 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.053257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.054158 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.054168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.054699 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.054703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.054929 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.055681 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:27.055703 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:27.057716 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:27.060042 (+ 2326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:27.060052 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:27.060075 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:27.060483 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:27.060862 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:27.060865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:27.060869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:27.061226 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:27.061857 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:27.061865 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:27.062398 (+ 533us) client_negotiation.cc:769] Sending connection context
0806 18:36:27.063902 (+ 1504us) client_negotiation.cc:240] Negotiation successful
0806 18:36:27.064364 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":161,"threads_started":1}
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:36:27.067461 31192 heartbeater.cc:347] Connected to a master server at 127.18.125.190:44167
I20260806 18:36:27.067746 31192 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:27.068461 31192 heartbeater.cc:511] Master 127.18.125.190:44167 requested a full tablet report, sending...
I20260806 18:36:27.069983 30819 ts_manager.cc:194] Registered new tserver with Master: 93fa408d88bb4ea9a091ebe01f8e9693 (127.18.125.130:34333)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
I20260806 18:36:27.070940 30819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:53545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:36:27.112403 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:38747/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44167
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:45617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:36:27 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041386, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
W20260806 18:36:27.349474 31197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:36:27.353745 31197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:36:27.353925 31197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:36:27.368511 31197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:36:27.368696 31197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:36:27.368841 31197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:36:27.387620 31197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:45617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.125.148:38747/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:36:27.389490 31197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:36:27.393432 31197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:36:27.436873 31203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:27.441785 31197 server_base.cc:1034] running on GCE node
W20260806 18:36:27.441954 31206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:36:27.448827 31202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:36:27.449599 31197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:36:27.450625 31197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:36:27.466180 31197 hybrid_clock.cc:648] HybridClock initialized: now 1786041387466058 us; error 154 us; skew 500 ppm
Aug 06 18:36:27 dist-test-slave-8w9v krb5kdc[30331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041387, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:36:27.479162 31197 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:36:27.480558 31197 webserver.cc:460] Webserver started at http://127.18.125.131:45685/ using document root <none> and password file <none>
I20260806 18:36:27.481426 31197 fs_manager.cc:359] Metadata directory not provided
I20260806 18:36:27.481600 31197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:36:27.481914 31197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:36:27 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041387, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:36:27.504922 31197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "4661b270746149e992f23234e491140a"
format_stamp: "Formatted at 2026-08-06 18:36:27 on dist-test-slave-8w9v"
server_key: "1ee5a65ced823d5157356147287abc63"
server_key_iv: "88a8b0f4b76ddad6854a4e51065fe284"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:27.507520 31197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "4661b270746149e992f23234e491140a"
format_stamp: "Formatted at 2026-08-06 18:36:27 on dist-test-slave-8w9v"
server_key: "1ee5a65ced823d5157356147287abc63"
server_key_iv: "88a8b0f4b76ddad6854a4e51065fe284"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:27.513973 31197 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260806 18:36:27.541566 31214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:27.543581 31197 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.001s
I20260806 18:36:27.543748 31197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "4661b270746149e992f23234e491140a"
format_stamp: "Formatted at 2026-08-06 18:36:27 on dist-test-slave-8w9v"
server_key: "1ee5a65ced823d5157356147287abc63"
server_key_iv: "88a8b0f4b76ddad6854a4e51065fe284"
server_key_version: "kuduclusterkey@0"
I20260806 18:36:27.543900 31197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:36:27.566692 31197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:36:27.570766 31197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:36:27.571069 31197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:36:27.571851 31197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:36:27.573037 31197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:36:27.573150 31197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:27.573247 31197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:36:27.573311 31197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:36:27.589745 31197 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:40813
I20260806 18:36:27.591117 31197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:36:27.591847 31338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:40813 every 8 connection(s)
I20260806 18:36:27.599200 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 31197
I20260806 18:36:27.599403 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:36:27.599743 18934 external_mini_cluster.cc:1428] Setting key 34cf8c76c7a8177b7d1f4b6d02509649
Aug 06 18:36:27 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041387, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:36:27.615792 31344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.596562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:48135 (local address 127.18.125.190:44167)
0806 18:36:27.599084 (+ 2522us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:27.599090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:27.599106 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:36:27.599155 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:27.599158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:27.599213 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:27.599307 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:27.602761 (+ 3454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.603593 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.605394 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.605755 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.605845 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.609151 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:27.609155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:27.609158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:27.609191 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:27.611036 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:27.611768 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:27.611772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:27.611774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:27.611898 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:27.614214 (+ 2316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:27.614218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:27.614220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:27.614494 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:27.614588 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:27.615419 (+ 831us) server_negotiation.cc:299] Negotiation successful
0806 18:36:27.615595 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2424,"thread_start_us":94,"threads_started":1}
I20260806 18:36:27.617731 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.595231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44167 (local address 127.18.125.131:48135)
0806 18:36:27.597404 (+ 2173us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:27.597467 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:27.598592 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:27.599660 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:27.599670 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:27.600050 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:36:27.602582 (+ 2532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.602604 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.603756 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.603760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.605228 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.605243 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.606080 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.606084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.606305 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.607096 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:36:27.607123 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:36:27.608910 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:36:27.611207 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:27.611215 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:27.611236 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:27.611631 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:27.612016 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:36:27.612019 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:36:27.612021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:36:27.614072 (+ 2051us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:36:27.614667 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:36:27.614676 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:36:27.615091 (+ 415us) client_negotiation.cc:769] Sending connection context
0806 18:36:27.616088 (+ 997us) client_negotiation.cc:240] Negotiation successful
0806 18:36:27.616532 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1917,"thread_start_us":151,"threads_started":1}
I20260806 18:36:27.619376 31339 heartbeater.cc:347] Connected to a master server at 127.18.125.190:44167
I20260806 18:36:27.619735 31339 heartbeater.cc:464] Registering TS with master...
I20260806 18:36:27.620532 31339 heartbeater.cc:511] Master 127.18.125.190:44167 requested a full tablet report, sending...
I20260806 18:36:27.622761 30819 ts_manager.cc:194] Registered new tserver with Master: 4661b270746149e992f23234e491140a (127.18.125.131:40813)
I20260806 18:36:27.623551 30819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:48135
I20260806 18:36:27.626025 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:36:27.642740 31344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.631719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.18.125.190:44167)
0806 18:36:27.631859 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:27.631864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:27.632693 (+ 829us) server_negotiation.cc:407] Connection header received
0806 18:36:27.632907 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:27.632911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:27.632966 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:27.633019 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:36:27.634078 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.634944 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.636254 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.636597 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.636679 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.638633 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:36:27.638637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:36:27.638639 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:36:27.638673 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:36:27.640507 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:27.641069 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:27.641073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:27.641075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:27.641189 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:36:27.641515 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:36:27.641519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:36:27.641521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:36:27.641761 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:36:27.641915 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:27.642385 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 18:36:27.642559 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:36:27.647166 30819 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:36:27.652294 30819 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:36:27.675007 31360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.665333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:40813 (local address 127.0.0.1:60488)
0806 18:36:27.667701 (+ 2368us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:27.667765 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:27.667929 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:27.668812 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:27.668820 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:27.668846 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:27.669016 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.669025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.671538 (+ 2513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.671542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.673274 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.673282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.674393 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.674397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.674534 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.674583 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 18:36:27.674683 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 18:36:27.674839 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2169,"mutex_wait_us":533,"thread_start_us":141,"threads_started":1}
I20260806 18:36:27.675766 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.665945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:37789 (local address 127.0.0.1:36128)
0806 18:36:27.668429 (+ 2484us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:27.668443 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:27.668534 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:27.669441 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:27.669447 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:27.669473 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:27.669608 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.669618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.671168 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.671172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.672676 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.672690 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.674186 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.674189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.675420 (+ 1231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.675454 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 18:36:27.675542 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 18:36:27.675625 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2420,"mutex_wait_us":1,"thread_start_us":120,"threads_started":1}
I20260806 18:36:27.676383 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.666306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36128 (local address 127.18.125.129:37789)
0806 18:36:27.668796 (+ 2490us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:27.668803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:27.668850 (+ 47us) server_negotiation.cc:407] Connection header received
0806 18:36:27.668940 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:27.668945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:27.669128 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:27.669271 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:27.669777 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.671024 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.672844 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.673799 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.673947 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.674106 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:27.676064 (+ 1958us) server_negotiation.cc:299] Negotiation successful
0806 18:36:27.676245 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2371,"thread_start_us":148,"threads_started":1}
I20260806 18:36:27.678224 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.665546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60488 (local address 127.18.125.131:40813)
0806 18:36:27.667793 (+ 2247us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:27.667810 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:27.667839 (+ 29us) server_negotiation.cc:407] Connection header received
0806 18:36:27.668175 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:27.668183 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:27.668437 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:27.668607 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:27.669172 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.671367 (+ 2195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.673434 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.674236 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.677709 (+ 3473us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.677901 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:27.677959 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 18:36:27.678072 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2112,"thread_start_us":121,"threads_started":1}
I20260806 18:36:27.680112 30969 tablet_service.cc:1467] Processing CreateTablet for tablet e4454238e6f04e84ba375e180896b639 (DEFAULT_TABLE table=test-table [id=b1b982d6ed694138a0dbd20c00834376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:27.681639 30969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4454238e6f04e84ba375e180896b639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:27.681610 31265 tablet_service.cc:1467] Processing CreateTablet for tablet e4454238e6f04e84ba375e180896b639 (DEFAULT_TABLE table=test-table [id=b1b982d6ed694138a0dbd20c00834376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:27.682829 31265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4454238e6f04e84ba375e180896b639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:27.683437 31360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.666673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:34333 (local address 127.0.0.1:38364)
0806 18:36:27.675687 (+ 9014us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:27.675702 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:27.676211 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:27.677360 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:27.677363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:27.677386 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:27.677469 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.677475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.679001 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.679005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.681333 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.681349 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.682906 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.682911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.683018 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.683051 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 18:36:27.683143 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 18:36:27.683247 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8948,"thread_start_us":103,"threads_started":1}
I20260806 18:36:27.684028 31361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.667128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.18.125.130:34333)
0806 18:36:27.668276 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:27.668283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:27.675795 (+ 7512us) server_negotiation.cc:407] Connection header received
0806 18:36:27.676398 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:27.676406 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:27.677028 (+ 622us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:27.677166 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:27.677634 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.678831 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.681539 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.682500 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.682656 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.682820 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:27.683735 (+ 915us) server_negotiation.cc:299] Negotiation successful
0806 18:36:27.683870 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1030,"thread_start_us":93,"threads_started":1}
I20260806 18:36:27.687589 31049 heartbeater.cc:503] Master 127.18.125.190:44167 was elected leader, sending a full tablet report...
I20260806 18:36:27.687736 31122 tablet_service.cc:1467] Processing CreateTablet for tablet e4454238e6f04e84ba375e180896b639 (DEFAULT_TABLE table=test-table [id=b1b982d6ed694138a0dbd20c00834376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:36:27.689113 31122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4454238e6f04e84ba375e180896b639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:36:27.695478 31362 tablet_bootstrap.cc:492] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: Bootstrap starting.
I20260806 18:36:27.697494 31364 tablet_bootstrap.cc:492] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: Bootstrap starting.
I20260806 18:36:27.699453 31362 tablet_bootstrap.cc:654] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:27.699977 31364 tablet_bootstrap.cc:654] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:27.700521 31362 log.cc:824] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:27.700906 31364 log.cc:824] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:27.701445 31363 tablet_bootstrap.cc:492] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6: Bootstrap starting.
I20260806 18:36:27.703365 31364 tablet_bootstrap.cc:492] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: No bootstrap required, opened a new log
I20260806 18:36:27.703459 31362 tablet_bootstrap.cc:492] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: No bootstrap required, opened a new log
I20260806 18:36:27.703596 31364 ts_tablet_manager.cc:1397] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:36:27.703639 31362 ts_tablet_manager.cc:1397] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 18:36:27.704044 31363 tablet_bootstrap.cc:654] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6: Neither blocks nor log segments found. Creating new log.
I20260806 18:36:27.704887 31363 log.cc:824] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:36:27.706911 31363 tablet_bootstrap.cc:492] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6: No bootstrap required, opened a new log
I20260806 18:36:27.707098 31363 ts_tablet_manager.cc:1397] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:36:27.707930 31364 raft_consensus.cc:348] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4661b270746149e992f23234e491140a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40813 } } peers { permanent_uuid: "3ddd5d279a83477db94942d3c3326ca6" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37789 } } peers { permanent_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34333 } }
I20260806 18:36:27.708042 31362 raft_consensus.cc:348] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4661b270746149e992f23234e491140a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40813 } } peers { permanent_uuid: "3ddd5d279a83477db94942d3c3326ca6" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37789 } } peers { permanent_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34333 } }
I20260806 18:36:27.708479 31362 raft_consensus.cc:374] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:27.708566 31364 raft_consensus.cc:374] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:27.708611 31362 raft_consensus.cc:729] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4661b270746149e992f23234e491140a, State: Initialized, Role: FOLLOWER
I20260806 18:36:27.708669 31364 raft_consensus.cc:729] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93fa408d88bb4ea9a091ebe01f8e9693, State: Initialized, Role: FOLLOWER
I20260806 18:36:27.709216 31364 consensus_queue.cc:260] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4661b270746149e992f23234e491140a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40813 } } peers { permanent_uuid: "3ddd5d279a83477db94942d3c3326ca6" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37789 } } peers { permanent_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34333 } }
I20260806 18:36:27.709249 31362 consensus_queue.cc:260] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4661b270746149e992f23234e491140a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40813 } } peers { permanent_uuid: "3ddd5d279a83477db94942d3c3326ca6" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37789 } } peers { permanent_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34333 } }
I20260806 18:36:27.710425 31364 ts_tablet_manager.cc:1428] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260806 18:36:27.710536 31362 ts_tablet_manager.cc:1428] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 18:36:27.710882 31192 heartbeater.cc:503] Master 127.18.125.190:44167 was elected leader, sending a full tablet report...
I20260806 18:36:27.710891 31339 heartbeater.cc:503] Master 127.18.125.190:44167 was elected leader, sending a full tablet report...
Aug 06 18:36:27 dist-test-slave-8w9v krb5kdc[30331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041386, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:36:27.714574 31363 raft_consensus.cc:348] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4661b270746149e992f23234e491140a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40813 } } peers { permanent_uuid: "3ddd5d279a83477db94942d3c3326ca6" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37789 } } peers { permanent_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34333 } }
I20260806 18:36:27.714902 31363 raft_consensus.cc:374] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:36:27.715016 31363 raft_consensus.cc:729] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ddd5d279a83477db94942d3c3326ca6, State: Initialized, Role: FOLLOWER
I20260806 18:36:27.715590 31363 consensus_queue.cc:260] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4661b270746149e992f23234e491140a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40813 } } peers { permanent_uuid: "3ddd5d279a83477db94942d3c3326ca6" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37789 } } peers { permanent_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34333 } }
I20260806 18:36:27.717061 31363 ts_tablet_manager.cc:1428] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260806 18:36:27.773914 31370 raft_consensus.cc:482] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:36:27.774195 31370 raft_consensus.cc:504] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4661b270746149e992f23234e491140a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40813 } } peers { permanent_uuid: "3ddd5d279a83477db94942d3c3326ca6" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37789 } } peers { permanent_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34333 } }
I20260806 18:36:27.775708 31370 leader_election.cc:288] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ddd5d279a83477db94942d3c3326ca6 (127.18.125.129:37789), 93fa408d88bb4ea9a091ebe01f8e9693 (127.18.125.130:34333)
I20260806 18:36:27.783166 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.776157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:37789 (local address 127.18.125.131:35445)
0806 18:36:27.777550 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:27.777568 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:27.777679 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:27.778242 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:27.778246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:27.778267 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:27.778391 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.778398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.779546 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.779552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.780912 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.780926 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.782415 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.782423 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.782635 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.782673 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:36:27.782780 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 18:36:27.782925 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822}
I20260806 18:36:27.785068 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.777259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:35445 (local address 127.18.125.129:37789)
0806 18:36:27.777413 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:27.777418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:27.777912 (+ 494us) server_negotiation.cc:407] Connection header received
0806 18:36:27.777990 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:27.777993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:27.778043 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:27.778115 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:27.778540 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.779348 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.781095 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.781736 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.781893 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.782228 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:27.784757 (+ 2529us) server_negotiation.cc:299] Negotiation successful
0806 18:36:27.784890 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:36:27.785897 31000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4454238e6f04e84ba375e180896b639" candidate_uuid: "4661b270746149e992f23234e491140a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ddd5d279a83477db94942d3c3326ca6" is_pre_election: true
I20260806 18:36:27.786317 31000 raft_consensus.cc:2454] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4661b270746149e992f23234e491140a in term 0.
I20260806 18:36:27.787859 31217 leader_election.cc:302] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ddd5d279a83477db94942d3c3326ca6, 4661b270746149e992f23234e491140a; no voters:
I20260806 18:36:27.788475 31370 raft_consensus.cc:2781] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:36:27.788583 31370 raft_consensus.cc:482] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:36:27.788651 31370 raft_consensus.cc:3037] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:27.790333 31370 raft_consensus.cc:504] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4661b270746149e992f23234e491140a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40813 } } peers { permanent_uuid: "3ddd5d279a83477db94942d3c3326ca6" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37789 } } peers { permanent_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34333 } }
I20260806 18:36:27.790851 31370 leader_election.cc:288] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [CANDIDATE]: Term 1 election: Requested vote from peers 3ddd5d279a83477db94942d3c3326ca6 (127.18.125.129:37789), 93fa408d88bb4ea9a091ebe01f8e9693 (127.18.125.130:34333)
I20260806 18:36:27.791801 31000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4454238e6f04e84ba375e180896b639" candidate_uuid: "4661b270746149e992f23234e491140a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ddd5d279a83477db94942d3c3326ca6"
I20260806 18:36:27.791992 31000 raft_consensus.cc:3037] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:27.793917 31000 raft_consensus.cc:2454] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4661b270746149e992f23234e491140a in term 1.
I20260806 18:36:27.794469 31217 leader_election.cc:302] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ddd5d279a83477db94942d3c3326ca6, 4661b270746149e992f23234e491140a; no voters:
I20260806 18:36:27.794741 31370 raft_consensus.cc:2781] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:36:27.795082 31370 raft_consensus.cc:686] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 1 LEADER]: Becoming Leader. State: Replica: 4661b270746149e992f23234e491140a, State: Running, Role: LEADER
I20260806 18:36:27.795591 31370 consensus_queue.cc:237] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4661b270746149e992f23234e491140a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40813 } } peers { permanent_uuid: "3ddd5d279a83477db94942d3c3326ca6" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37789 } } peers { permanent_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34333 } }
I20260806 18:36:27.796947 31361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.777428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:32983 (local address 127.18.125.130:34333)
0806 18:36:27.777557 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:27.777562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:27.783618 (+ 6056us) server_negotiation.cc:407] Connection header received
0806 18:36:27.786060 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:27.786065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:27.786129 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:27.786182 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:36:27.786641 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.787412 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.793226 (+ 5814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.794055 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.794193 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.794297 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:27.796625 (+ 2328us) server_negotiation.cc:299] Negotiation successful
0806 18:36:27.796759 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:36:27.797662 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.776443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:34333 (local address 127.18.125.131:32983)
0806 18:36:27.783494 (+ 7051us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:27.783519 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 18:36:27.785824 (+ 2305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:36:27.786352 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:36:27.786355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:36:27.786379 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:36:27.786489 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.786496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.791499 (+ 5003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.791503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.792937 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:36:27.793018 (+ 81us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.794424 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.794428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:36:27.794540 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.794594 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 18:36:27.797370 (+ 2776us) client_negotiation.cc:240] Negotiation successful
0806 18:36:27.797483 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6950,"thread_start_us":133,"threads_started":1}
W20260806 18:36:27.798187 31193 tablet.cc:2369] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:36:27.800113 31130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4454238e6f04e84ba375e180896b639" candidate_uuid: "4661b270746149e992f23234e491140a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" is_pre_election: true
I20260806 18:36:27.800462 31130 raft_consensus.cc:2454] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4661b270746149e992f23234e491140a in term 0.
I20260806 18:36:27.800803 31144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4454238e6f04e84ba375e180896b639" candidate_uuid: "4661b270746149e992f23234e491140a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93fa408d88bb4ea9a091ebe01f8e9693"
I20260806 18:36:27.801050 31144 raft_consensus.cc:3037] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:36:27.802718 31144 raft_consensus.cc:2454] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4661b270746149e992f23234e491140a in term 1.
I20260806 18:36:27.807363 30819 catalog_manager.cc:5495] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4661b270746149e992f23234e491140a (127.18.125.131). New cstate: current_term: 1 leader_uuid: "4661b270746149e992f23234e491140a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4661b270746149e992f23234e491140a" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 40813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ddd5d279a83477db94942d3c3326ca6" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34333 } health_report { overall_health: UNKNOWN } } }
W20260806 18:36:27.843231 31340 tablet.cc:2369] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:36:27.844379 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.836844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.18.125.131:40813)
0806 18:36:27.837028 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 18:36:27.837036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:36:27.837310 (+ 274us) server_negotiation.cc:407] Connection header received
0806 18:36:27.837439 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:36:27.837443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:36:27.837525 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:36:27.837653 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:36:27.838162 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.838983 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.840338 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:36:27.840733 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:36:27.840863 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:36:27.841336 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:36:27.842209 (+ 873us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:36:27.842406 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 18:36:27.842886 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 18:36:27.843133 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:36:27.852813 31144 raft_consensus.cc:1261] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693 [term 1 FOLLOWER]: Refusing update from remote peer 4661b270746149e992f23234e491140a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:36:27.853842 31000 raft_consensus.cc:1261] T e4454238e6f04e84ba375e180896b639 P 3ddd5d279a83477db94942d3c3326ca6 [term 1 FOLLOWER]: Refusing update from remote peer 4661b270746149e992f23234e491140a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:36:27.855032 31375 consensus_queue.cc:1035] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [LEADER]: Connected to new peer: Peer: permanent_uuid: "93fa408d88bb4ea9a091ebe01f8e9693" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 34333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:36:27.855247 31370 consensus_queue.cc:1035] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ddd5d279a83477db94942d3c3326ca6" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:36:27.871941 31378 mvcc.cc:205] Tried to move back new op lower bound from 7315625524634275840 to 7315625524445458432. Current Snapshot: MvccSnapshot[applied={T|T < 7315625524634275840}]
I20260806 18:36:27.874688 31379 mvcc.cc:205] Tried to move back new op lower bound from 7315625524634275840 to 7315625524445458432. Current Snapshot: MvccSnapshot[applied={T|T < 7315625524634275840}]
I20260806 18:36:27.882332 31381 mvcc.cc:205] Tried to move back new op lower bound from 7315625524634275840 to 7315625524445458432. Current Snapshot: MvccSnapshot[applied={T|T < 7315625524634275840}]
I20260806 18:36:27.888640 30819 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:36:27.888854 30819 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:36:27.891995 30819 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c782ab79490e4df2a581a69f3fab230b: Sending DeleteTablet for 3 replicas of tablet e4454238e6f04e84ba375e180896b639
I20260806 18:36:27.893059 31265 tablet_service.cc:1514] Processing DeleteTablet for tablet e4454238e6f04e84ba375e180896b639 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:27 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:60488
I20260806 18:36:27.894384 30969 tablet_service.cc:1514] Processing DeleteTablet for tablet e4454238e6f04e84ba375e180896b639 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:27 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:36128
I20260806 18:36:27.894945 31122 tablet_service.cc:1514] Processing DeleteTablet for tablet e4454238e6f04e84ba375e180896b639 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:36:27 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:38364
I20260806 18:36:27.896523 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 30913
I20260806 18:36:27.902925 31391 tablet_replica.cc:330] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: stopping tablet replica
I20260806 18:36:27.904611 31391 raft_consensus.cc:2229] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:36:27.905216 31391 raft_consensus.cc:2258] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:36:27.908851 31389 tablet_replica.cc:330] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: stopping tablet replica
I20260806 18:36:27.910712 31391 ts_tablet_manager.cc:1905] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:36:27.911095 31389 raft_consensus.cc:2229] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:36:27.911631 31389 raft_consensus.cc:2258] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:36:27.915092 31391 ts_tablet_manager.cc:1918] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:36:27.920506 31389 ts_tablet_manager.cc:1905] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:36:27.924023 31389 ts_tablet_manager.cc:1918] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:36:27.924404 31391 log.cc:1196] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/e4454238e6f04e84ba375e180896b639
I20260806 18:36:27.924901 31391 ts_tablet_manager.cc:1939] T e4454238e6f04e84ba375e180896b639 P 93fa408d88bb4ea9a091ebe01f8e9693: Deleting consensus metadata
W20260806 18:36:27.925496 30812 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:37789 (error 108)
I20260806 18:36:27.928324 30812 catalog_manager.cc:4841] TS 93fa408d88bb4ea9a091ebe01f8e9693 (127.18.125.130:34333): tablet e4454238e6f04e84ba375e180896b639 (table test-table [id=b1b982d6ed694138a0dbd20c00834376]) successfully deleted
I20260806 18:36:27.928843 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 31057
I20260806 18:36:27.936336 31389 log.cc:1196] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/e4454238e6f04e84ba375e180896b639
I20260806 18:36:27.939291 31389 ts_tablet_manager.cc:1939] T e4454238e6f04e84ba375e180896b639 P 4661b270746149e992f23234e491140a: Deleting consensus metadata
I20260806 18:36:27.941075 30811 catalog_manager.cc:4841] TS 4661b270746149e992f23234e491140a (127.18.125.131:40813): tablet e4454238e6f04e84ba375e180896b639 (table test-table [id=b1b982d6ed694138a0dbd20c00834376]) successfully deleted
I20260806 18:36:27.948032 31360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:36:27.947614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:37789 (local address 127.0.0.1:36130)
0806 18:36:27.947835 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 18:36:27.947918 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:37789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260806 18:36:27.948416 30812 catalog_manager.cc:4584] TS 3ddd5d279a83477db94942d3c3326ca6 (127.18.125.129:37789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4454238e6f04e84ba375e180896b639: Network error: Client connection negotiation failed: client connection to 127.18.125.129:37789: connect: Connection refused (error 111)
I20260806 18:36:27.949623 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 31197
I20260806 18:36:27.959023 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 30794
I20260806 18:36:27.967316 18934 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 18:36:28.313499 18934 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 18:36:28.313697 18934 mini_ranger.cc:66] Stopping Ranger...
I20260806 18:36:28.659672 18934 mini_ranger.cc:68] Stopped Ranger
2026-08-06 18:36:28.659 UTC [30363] LOG: received smart shutdown request
2026-08-06 18:36:28.664 UTC [30363] LOG: background worker "logical replication launcher" (PID 30373) exited with exit code 1
2026-08-06 18:36:28.664 UTC [30368] LOG: shutting down
2026-08-06 18:36:28.874 UTC [30363] LOG: database system is shut down
2026-08-06T18:36:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53897 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:36:29 dist-test-slave-8w9v krb5kdc[31403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:36:29 dist-test-slave-8w9v krb5kdc[31403](info): set up 2 sockets
Aug 06 18:36:29 dist-test-slave-8w9v krb5kdc[31403](info): commencing operation
krb5kdc: starting...
W20260806 18:36:30.146735 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:36:30 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:36:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:36:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/rangeradmin_127.18.125.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/rangeradmin_127.18.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/rangerlookup_127.18.125.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/rangerlookup_127.18.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/HTTP_127.18.125.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/HTTP_127.18.125.148@KRBTEST.COM.keytab.
I20260806 18:36:30.347807 18934 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskun4LBX/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/postgres -l logfile start
2026-08-06 18:36:33.315 UTC [31436] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 18:36:33.316 UTC [31436] LOG: listening on IPv4 address "127.18.125.148", port 51693
2026-08-06 18:36:33.320 UTC [31436] LOG: listening on Unix socket "/tmp/.s.PGSQL.51693"
2026-08-06 18:36:33.344 UTC [31439] LOG: database system was shut down at 2026-08-06 18:36:31 UTC
2026-08-06 18:36:33.348 UTC [31436] LOG: database system is ready to accept connections
I20260806 18:36:34.350906 18934 mini_postgres.cc:96] Postgres bound to 51693
2026-08-06 18:36:34.358 UTC [31447] FATAL: database "slave" does not exist
127.18.125.148:51693 - accepting connections
I20260806 18:36:34.359248 18934 mini_ranger.cc:161] Starting Ranger...
I20260806 18:36:34.376387 18934 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 18:36:35.105069 18934 mini_ranger.cc:87] Created ranger Postgres database
I20260806 18:36:35.105176 18934 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-admin
2026-08-06 18:36:35,418 [I] DB FLAVOR :POSTGRES
2026-08-06 18:36:35,418 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:36:35,418 [I] Checking connection..
2026-08-06 18:36:35,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 18:36:35,700 [I] Checking connection passed.
2026-08-06 18:36:35,701 [I] --------- Verifying version history table ---------
2026-08-06 18:36:35,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:36:35,988 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:36:35,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:36:36,277 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:36:36,277 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 18:36:36,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 18:36:36.580 UTC [31512] WARNING: there is no transaction in progress
2026-08-06 18:36:36,588 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 18:36:36,589 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:36:36,880 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 18:36:36,880 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:36:36,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 18:36:37,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 18:36:37,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8w9v.c.gcp-upstream.internal','N') ;"
2026-08-06 18:36:37,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:36:38,012 [I] Table x_portal_user does not exist in database ranger
2026-08-06 18:36:38,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:36:38,304 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 18:36:38,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:36:38,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:36:38,890 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 18:36:38,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 18:36:40.065 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.085 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.105 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.123 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.143 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.165 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.244 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.253 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.277 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.289 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.301 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.317 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.330 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.338 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.346 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:40.355 UTC [31624] WARNING: there is no transaction in progress
2026-08-06 18:36:41,236 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 18:36:41,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 18:36:41,522 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 18:36:41,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:36:41,781 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 18:36:41,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:36:42,046 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 18:36:42,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:36:42,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:36:42,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8w9v.c.gcp-upstream.internal';"
2026-08-06 18:36:42,900 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 18:36:42,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 18:36:43,178 [I] DB_PATCHES have already been applied
I20260806 18:36:43.185395 18934 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-admin:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskun4LBX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 18:36:43.185475 18934 mini_ranger.cc:193] Using host: 127.18.125.148
I20260806 18:36:43.189936 18934 mini_ranger.cc:227] Ranger admin URL: http://127.18.125.148:36449
Aug 06, 2026 6:36:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:36:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:36:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:36:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:36:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:36:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:36:44 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 18:36:44.379771 18934 mini_ranger.cc:160] Time spent starting Ranger: real 10.021s user 0.000s sys 0.012s
Aug 06, 2026 6:37:07 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:37:07 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 06, 2026 6:37:07 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:37:07 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:37:08 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 18:37:08.743808 18934 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.125.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.125.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/rangerkms_127.18.125.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.125.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/rangerkms_127.18.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.125.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.125.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.125.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/HTTP_127.18.125.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.125.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/HTTP_127.18.125.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 18:37:08 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041428, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 18:37:08.845162 18934 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 18:37:08.856165 18934 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 18:37:09.961330 18934 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 18:37:09.961453 18934 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-kms
I20260806 18:37:09.961478 18934 mini_ranger_kms.cc:227] Using postgres at 127.18.125.148:51693
2026-08-06 18:37:10,155 [I] DB FLAVOR :POSTGRES
2026-08-06 18:37:10,155 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:37:10,155 [I] Checking connection
2026-08-06 18:37:10,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 18:37:10,451 [I] connection success
2026-08-06 18:37:10,451 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 18:37:10,451 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 18:37:10,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 18:37:10,721 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 18:37:10,721 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:37:10,721 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 18:37:10,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.125.148:51693/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 18:37:11,011 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 18:37:11.557651 18934 mini_ranger_kms.cc:310] Created kms service
I20260806 18:37:11.783267 18934 mini_ranger_kms.cc:326] Created kudu user
I20260806 18:37:11.840878 18934 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 18:37:11 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041428, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:37:12.287688 18934 mini_ranger_kms.cc:384] Added ranger policy
I20260806 18:37:12.287853 18934 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-kms:/tmp/dist-test-taskun4LBX/build/debug/bin/postgresql.jar:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskun4LBX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 18:37:12.287874 18934 mini_ranger_kms.cc:240] Using host: 127.18.125.148
I20260806 18:37:12.292145 18934 mini_ranger_kms.cc:276] Ranger KMS PID: 31814
I20260806 18:37:12.292236 18934 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.125.148:54461
Aug 06, 2026 6:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:37:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 18:37:14 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041434, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.125.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:14.760618 18934 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.915s user 0.018s sys 0.158s
I20260806 18:37:14.760751 18934 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 18:37:14.760816 18934 mini_ranger_kms.cc:401] 127.18.125.148:54461/kms/v1/keys
Aug 06 18:37:15 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041434, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.125.148@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
Aug 06 18:37:15 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041434, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.125.148@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
Aug 06 18:37:18 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:37:18.168417 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34319
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:34319
--ranger_config_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:54461/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:18.302687 31868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:18.303027 31868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:18.303071 31868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:18.307221 31868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:18.307291 31868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:18.307312 31868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:18.307328 31868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:18.307344 31868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:18.313105 31868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.125.148:54461/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:34319
--ranger_config_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:18.314333 31868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:18.315382 31868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:18.324966 31876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:18.326604 31875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:18.326617 31878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:18.326694 31868 server_base.cc:1034] running on GCE node
I20260806 18:37:18.327365 31868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:18.328714 31868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:18.329885 31868 hybrid_clock.cc:648] HybridClock initialized: now 1786041438329874 us; error 33 us; skew 500 ppm
Aug 06 18:37:18 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:18.336210 31868 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:37:18.337229 31868 webserver.cc:460] Webserver started at http://127.18.125.190:39107/ using document root <none> and password file <none>
I20260806 18:37:18.337826 31868 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:18.337886 31868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:18.338121 31868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:37:18 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:37:18.389647 31868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088"
format_stamp: "Formatted at 2026-08-06 18:37:18 on dist-test-slave-8w9v"
server_key: "f5f1e156de5f037b3a78e8dd45388b77"
server_key_iv: "f636477ef4ec721cb93ff5c3602214b4"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:18.390347 31868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088"
format_stamp: "Formatted at 2026-08-06 18:37:18 on dist-test-slave-8w9v"
server_key: "f5f1e156de5f037b3a78e8dd45388b77"
server_key_iv: "f636477ef4ec721cb93ff5c3602214b4"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:18.394691 31868 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:37:18.410651 31885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:18.412271 31868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 18:37:18.412441 31868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088"
format_stamp: "Formatted at 2026-08-06 18:37:18 on dist-test-slave-8w9v"
server_key: "f5f1e156de5f037b3a78e8dd45388b77"
server_key_iv: "f636477ef4ec721cb93ff5c3602214b4"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:18.412587 31868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:18.427201 31868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:18.431092 31868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:18.431310 31868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:18.440455 31868 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:34319
I20260806 18:37:18.440461 31938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:34319 every 8 connection(s)
I20260806 18:37:18.441587 31868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:37:18.445474 31939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:18.448273 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 31868
I20260806 18:37:18.448451 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:37:18.448743 18934 external_mini_cluster.cc:1428] Setting key dfdbcb7cf47529511052c2f76f12a15d
I20260806 18:37:18.453867 31939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088: Bootstrap starting.
I20260806 18:37:18.457167 31939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088: Neither blocks nor log segments found. Creating new log.
Aug 06 18:37:18 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:18.458329 31939 log.cc:824] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:18.461202 31939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088: No bootstrap required, opened a new log
I20260806 18:37:18.464875 31942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:18.450318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.18.125.190:34319)
0806 18:37:18.450763 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:18.450780 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:18.450823 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:37:18.451485 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:18.451502 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:18.451852 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:18.452232 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:18.453085 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:18.454145 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:18.455018 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:18.455500 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:18.455677 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:18.458378 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:18.458383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:18.458398 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:18.458453 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:18.461232 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:18.461876 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:18.461882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:18.461888 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:18.462024 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:18.462401 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:18.462404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:18.462406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:18.462941 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:18.463156 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:18.463516 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 18:37:18.463815 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":146,"threads_started":1}
I20260806 18:37:18.465404 31939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } }
I20260806 18:37:18.465679 31939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:18.465727 31939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f5fbd1ffd0a405ca92d6cbb14b4d088, State: Initialized, Role: FOLLOWER
I20260806 18:37:18.466285 31939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } }
I20260806 18:37:18.466418 31939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:37:18.466485 31939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:37:18.466585 31939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:18.467765 31939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } }
I20260806 18:37:18.468180 31939 leader_election.cc:302] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f5fbd1ffd0a405ca92d6cbb14b4d088; no voters:
I20260806 18:37:18.468571 31939 leader_election.cc:288] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:37:18.468663 31944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:18.468879 31944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 1 LEADER]: Becoming Leader. State: Replica: 9f5fbd1ffd0a405ca92d6cbb14b4d088, State: Running, Role: LEADER
I20260806 18:37:18.469300 31944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } }
I20260806 18:37:18.469947 31939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:18.470570 31946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f5fbd1ffd0a405ca92d6cbb14b4d088. Latest consensus state: current_term: 1 leader_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } } }
I20260806 18:37:18.470708 31946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:18.471143 31950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:37:18.471478 31945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } } }
I20260806 18:37:18.471624 31945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:18.475142 31950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:37:18.475636 31939 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:37:18.487860 31950 catalog_manager.cc:1347] Generated new cluster ID: d891f43e9466464e847278eafc66cb55
I20260806 18:37:18.488011 31950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:37:18.495608 31950 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:37:18.496956 31950 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:37:18.501549 31950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088: Generated new TSK 0
I20260806 18:37:18.502321 31950 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:37:19.296248 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:54461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34319
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:19.452137 31986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:19.452598 31986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:19.452699 31986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:19.458173 31986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:19.458293 31986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:19.458413 31986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:37:19.465026 31986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.125.148:54461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:19.466250 31986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:19.467414 31986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:37:19.479861 31986 server_base.cc:1034] running on GCE node
W20260806 18:37:19.479768 31994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:19.480115 31997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:19.480369 31995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:19.480929 31986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:19.481855 31986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:19.483729 31986 hybrid_clock.cc:648] HybridClock initialized: now 1786041439483609 us; error 100 us; skew 500 ppm
Aug 06 18:37:19 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:19.488451 31986 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:37:19.489464 31986 webserver.cc:460] Webserver started at http://127.18.125.129:40325/ using document root <none> and password file <none>
I20260806 18:37:19.490127 31986 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:19.490227 31986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:19.490486 31986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:37:19 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:37:19.509560 31986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "3c5c00de630345cfa531883548165618"
format_stamp: "Formatted at 2026-08-06 18:37:19 on dist-test-slave-8w9v"
server_key: "a04a18f0082eef9f004f14f589377b2c"
server_key_iv: "e83797773af327b6b09fd3e41c0136d7"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:19.510947 31986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "3c5c00de630345cfa531883548165618"
format_stamp: "Formatted at 2026-08-06 18:37:19 on dist-test-slave-8w9v"
server_key: "a04a18f0082eef9f004f14f589377b2c"
server_key_iv: "e83797773af327b6b09fd3e41c0136d7"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:19.526001 31986 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.004s sys 0.000s
I20260806 18:37:19.550374 32005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:19.552280 31986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:19.552456 31986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "3c5c00de630345cfa531883548165618"
format_stamp: "Formatted at 2026-08-06 18:37:19 on dist-test-slave-8w9v"
server_key: "a04a18f0082eef9f004f14f589377b2c"
server_key_iv: "e83797773af327b6b09fd3e41c0136d7"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:19.552627 31986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:19.579295 31986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:19.583703 31986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:19.584019 31986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:19.584949 31986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:19.586187 31986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:19.586318 31986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:19.586689 31986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:19.586793 31986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:19.601799 31986 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:37465
I20260806 18:37:19.603250 31986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:37:19.605213 32127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:37465 every 8 connection(s)
I20260806 18:37:19.606124 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 31986
I20260806 18:37:19.606261 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:37:19.606585 18934 external_mini_cluster.cc:1428] Setting key 8a6032da2204c5b52a653edfa31d5106
Aug 06 18:37:19 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:19.627123 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:19.608225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:45507 (local address 127.18.125.190:34319)
0806 18:37:19.610631 (+ 2406us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:19.610636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:19.612455 (+ 1819us) server_negotiation.cc:407] Connection header received
0806 18:37:19.612511 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:19.612514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:19.612574 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:19.612669 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:19.615798 (+ 3129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:19.616775 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:19.618687 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:19.619049 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:19.619131 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:19.622472 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:19.622477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:19.622480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:19.622511 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:19.624520 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:19.625192 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:19.625197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:19.625199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:19.625313 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:19.625740 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:19.625744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:19.625746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:19.625975 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:19.626046 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:19.626786 (+ 740us) server_negotiation.cc:299] Negotiation successful
0806 18:37:19.626932 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2311,"thread_start_us":142,"threads_started":1}
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:37:19.629132 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:19.608932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34319 (local address 127.18.125.129:45507)
0806 18:37:19.610926 (+ 1994us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:19.610989 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:19.612098 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:19.614466 (+ 2368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:19.614476 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:19.614871 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:19.615603 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:19.615625 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:19.616952 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:19.616956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:19.618529 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:19.618541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:19.619384 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:19.619389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:19.619607 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:19.620400 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:19.620422 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:19.622230 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:19.624675 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:19.624682 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:19.624699 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:19.625061 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:19.625440 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:19.625443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:19.625445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:19.625627 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:19.626131 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:19.626138 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:19.626485 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 18:37:19.627653 (+ 1168us) client_negotiation.cc:240] Negotiation successful
0806 18:37:19.628022 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1748,"thread_start_us":154,"threads_started":1}
I20260806 18:37:19.630697 32128 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34319
I20260806 18:37:19.630946 32128 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:19.631534 32128 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
I20260806 18:37:19.633369 31903 ts_manager.cc:194] Registered new tserver with Master: 3c5c00de630345cfa531883548165618 (127.18.125.129:37465)
I20260806 18:37:19.635082 31903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:45507
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:37:19.653856 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:54461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34319
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:37:19 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 18:37:19.829677 32134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:19.830123 32134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:19.830211 32134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:19.836790 32134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:19.836957 32134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:19.837087 32134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:37:19.845366 32134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.125.148:54461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:19.846843 32134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:19.848655 32134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:19.862159 32141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:19.862154 32143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:19.862707 32134 server_base.cc:1034] running on GCE node
W20260806 18:37:19.863691 32140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:19.864444 32134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:19.865432 32134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:19.866822 32134 hybrid_clock.cc:648] HybridClock initialized: now 1786041439866751 us; error 95 us; skew 500 ppm
Aug 06 18:37:19 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:19.871438 32134 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:37:19.873008 32134 webserver.cc:460] Webserver started at http://127.18.125.130:35207/ using document root <none> and password file <none>
I20260806 18:37:19.873855 32134 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:19.873999 32134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:19.874328 32134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:37:19 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:37:19.892028 32134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "29c7d42bc9714223add03117ddb3e59a"
format_stamp: "Formatted at 2026-08-06 18:37:19 on dist-test-slave-8w9v"
server_key: "eb939d085a505e1609cb676d857c4a32"
server_key_iv: "cf3bfbc29016d831009e99ee38860a7c"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:19.893055 32134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "29c7d42bc9714223add03117ddb3e59a"
format_stamp: "Formatted at 2026-08-06 18:37:19 on dist-test-slave-8w9v"
server_key: "eb939d085a505e1609cb676d857c4a32"
server_key_iv: "cf3bfbc29016d831009e99ee38860a7c"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:19.898938 32134 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 18:37:19.918186 32150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:19.920112 32134 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:37:19.920301 32134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "29c7d42bc9714223add03117ddb3e59a"
format_stamp: "Formatted at 2026-08-06 18:37:19 on dist-test-slave-8w9v"
server_key: "eb939d085a505e1609cb676d857c4a32"
server_key_iv: "cf3bfbc29016d831009e99ee38860a7c"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:19.920502 32134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:19.973809 32134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:19.978250 32134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:19.978561 32134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:19.979465 32134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:19.980686 32134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:19.980796 32134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:19.980901 32134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:19.980979 32134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:19.995296 32134 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:39825
I20260806 18:37:19.998826 32269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:39825 every 8 connection(s)
I20260806 18:37:20.000344 32134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:37:20.008360 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 32134
I20260806 18:37:20.008502 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:37:20.008841 18934 external_mini_cluster.cc:1428] Setting key c1b9b722707a743c23e14d47af566018
Aug 06 18:37:20 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:20.028609 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:20.006314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:35263 (local address 127.18.125.190:34319)
0806 18:37:20.006501 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:20.006506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:20.012757 (+ 6251us) server_negotiation.cc:407] Connection header received
0806 18:37:20.014298 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:20.014302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:20.014376 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:20.014476 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:20.016338 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:20.017198 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:20.019207 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:20.019586 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:20.019677 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:20.023515 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:20.023519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:20.023522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:20.023558 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:20.025605 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:20.026362 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:20.026366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:20.026367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:20.026467 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:20.027070 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:20.027074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:20.027075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:20.027348 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:20.027445 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:20.028266 (+ 821us) server_negotiation.cc:299] Negotiation successful
0806 18:37:20.028433 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:37:20.030872 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:20.006917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34319 (local address 127.18.125.130:35263)
0806 18:37:20.012531 (+ 5614us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:20.012617 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:20.013974 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:20.014832 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:20.014843 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:20.015267 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:20.016104 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:20.016155 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:20.017607 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:20.017610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:20.019049 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:20.019061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:20.019925 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:20.019929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:20.020273 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:20.021056 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:20.021079 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:20.023221 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:20.025774 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:20.025781 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:20.025805 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:20.026229 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:20.026735 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:20.026738 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:20.026740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:20.026939 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:20.027562 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:20.027569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:20.027920 (+ 351us) client_negotiation.cc:769] Sending connection context
0806 18:37:20.029260 (+ 1340us) client_negotiation.cc:240] Negotiation successful
0806 18:37:20.029681 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5301,"thread_start_us":176,"threads_started":1}
I20260806 18:37:20.032716 32270 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34319
I20260806 18:37:20.032997 32270 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:20.033649 32270 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
I20260806 18:37:20.035181 31903 ts_manager.cc:194] Registered new tserver with Master: 29c7d42bc9714223add03117ddb3e59a (127.18.125.130:39825)
I20260806 18:37:20.036007 31903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:35263
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:37:20.069237 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:54461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34319
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:37:20 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041439, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
W20260806 18:37:20.278522 32275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:20.279424 32275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:20.279551 32275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:20.285641 32275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:20.285864 32275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:20.286003 32275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:37:20.294230 32275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.125.148:54461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:20.296037 32275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:20.297595 32275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:20.311178 32281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:20.312006 32280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:20.312603 32275 server_base.cc:1034] running on GCE node
W20260806 18:37:20.311118 32283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:20.316780 32275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:20.317749 32275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:20.333279 32275 hybrid_clock.cc:648] HybridClock initialized: now 1786041440333165 us; error 148 us; skew 500 ppm
Aug 06 18:37:20 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:20.338083 32275 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:37:20.339540 32275 webserver.cc:460] Webserver started at http://127.18.125.131:42011/ using document root <none> and password file <none>
I20260806 18:37:20.340530 32275 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:20.340693 32275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:20.341022 32275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:37:20 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:37:20.364991 32275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "44ff315422f24904805a37566201ed16"
format_stamp: "Formatted at 2026-08-06 18:37:20 on dist-test-slave-8w9v"
server_key: "70363c52ba10745b13736ca7adb9d587"
server_key_iv: "0a7cd3eef99f23967307e67dd3644506"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:20.366093 32275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "44ff315422f24904805a37566201ed16"
format_stamp: "Formatted at 2026-08-06 18:37:20 on dist-test-slave-8w9v"
server_key: "70363c52ba10745b13736ca7adb9d587"
server_key_iv: "0a7cd3eef99f23967307e67dd3644506"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:20.382189 32275 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.001s sys 0.004s
I20260806 18:37:20.403173 32290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:20.405369 32275 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.002s
I20260806 18:37:20.405570 32275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "44ff315422f24904805a37566201ed16"
format_stamp: "Formatted at 2026-08-06 18:37:20 on dist-test-slave-8w9v"
server_key: "70363c52ba10745b13736ca7adb9d587"
server_key_iv: "0a7cd3eef99f23967307e67dd3644506"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:20.405752 32275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:20.449954 32275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:20.454622 32275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:20.454941 32275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:20.455745 32275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:20.457120 32275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:20.457242 32275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 18:37:20.457355 32275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:20.457423 32275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:20.471083 32275 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:44185
I20260806 18:37:20.472559 32275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:37:20.477427 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 32275
I20260806 18:37:20.477608 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:37:20.478041 18934 external_mini_cluster.cc:1428] Setting key 5a1c1678903a5e713959468d8793ffad
I20260806 18:37:20.480852 32408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:44185 every 8 connection(s)
Aug 06 18:37:20 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:20.499644 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:20.484868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:38313 (local address 127.18.125.190:34319)
0806 18:37:20.485004 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:20.485009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:20.485903 (+ 894us) server_negotiation.cc:407] Connection header received
0806 18:37:20.487271 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:20.487277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:20.487355 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:20.487415 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:20.489214 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:20.490119 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:20.491923 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:20.492277 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:20.492383 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:20.495538 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:20.495542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:20.495544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:20.495572 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:20.497278 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:20.497903 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:20.497913 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:20.497914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:20.498000 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:20.498396 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:20.498400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:20.498403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:20.498611 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:20.498732 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:20.499331 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 18:37:20.499471 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:37:20.500679 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:20.483525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34319 (local address 127.18.125.131:38313)
0806 18:37:20.485712 (+ 2187us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:20.485780 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:20.486980 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:20.487881 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:20.487891 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:20.488324 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:20.489026 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:20.489046 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:20.490316 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:20.490319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:20.491756 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:20.491770 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:20.492633 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:20.492637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:20.492871 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:20.493704 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:20.493728 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:20.495309 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:20.497431 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:20.497438 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:20.497459 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:20.497766 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:20.498104 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:20.498108 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:20.498110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:20.498285 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:20.498732 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:20.498741 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:20.499052 (+ 311us) client_negotiation.cc:769] Sending connection context
0806 18:37:20.499280 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 18:37:20.499636 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1943,"thread_start_us":201,"threads_started":1}
I20260806 18:37:20.502111 32409 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34319
I20260806 18:37:20.502373 32409 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:20.502930 32409 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
I20260806 18:37:20.504263 31902 ts_manager.cc:194] Registered new tserver with Master: 44ff315422f24904805a37566201ed16 (127.18.125.131:44185)
I20260806 18:37:20.504884 31902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:38313
I20260806 18:37:20.506330 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:37:20.507184 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 31986
I20260806 18:37:20.515781 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 32134
I20260806 18:37:20.525449 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 32275
I20260806 18:37:20.534570 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 31868
I20260806 18:37:20.545054 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34319
--webserver_interface=127.18.125.190
--webserver_port=39107
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:34319
--ranger_config_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:54461/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.125.148:54461/kms with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:20.665215 32412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:20.665585 32412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:20.665665 32412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:20.670020 32412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:20.670145 32412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:20.670186 32412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:20.670260 32412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:20.670306 32412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:20.676491 32412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.125.148:54461/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:34319
--ranger_config_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=39107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:20.677800 32412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:20.678972 32412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:37:20.688928 32412 server_base.cc:1034] running on GCE node
W20260806 18:37:20.688871 32417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:20.688862 32420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:20.688862 32418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:20.689877 32412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:20.691089 32412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:20.692230 32412 hybrid_clock.cc:648] HybridClock initialized: now 1786041440692199 us; error 37 us; skew 500 ppm
Aug 06 18:37:20 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:20.695359 32412 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:37:20.696573 32412 webserver.cc:460] Webserver started at http://127.18.125.190:39107/ using document root <none> and password file <none>
I20260806 18:37:20.697233 32412 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:20.697320 32412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:37:20 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:37:20.785544 32412 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:20.788149 32428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:20.789623 32412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:20.789798 32412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088"
format_stamp: "Formatted at 2026-08-06 18:37:18 on dist-test-slave-8w9v"
server_key: "f5f1e156de5f037b3a78e8dd45388b77"
server_key_iv: "f636477ef4ec721cb93ff5c3602214b4"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:20.790454 32412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:20.803819 32412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:20.807533 32412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:20.807822 32412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:20.817358 32412 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:34319
I20260806 18:37:20.817410 32482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:34319 every 8 connection(s)
I20260806 18:37:20.818652 32412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:37:20.823463 32483 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:37:20.824919 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 32412
I20260806 18:37:20.825416 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:54461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:37465
--local_ip_for_outbound_sockets=127.18.125.129
--tserver_master_addrs=127.18.125.190:34319
--webserver_port=40325
--webserver_interface=127.18.125.129
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.125.148:54461/kms with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:37:20.825410 32483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088: Bootstrap starting.
I20260806 18:37:20.844487 32483 log.cc:824] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:20.852344 32483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:37:20.852818 32483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088: Bootstrap complete.
I20260806 18:37:20.857421 32483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } }
I20260806 18:37:20.857925 32483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f5fbd1ffd0a405ca92d6cbb14b4d088, State: Initialized, Role: FOLLOWER
I20260806 18:37:20.858448 32483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } }
I20260806 18:37:20.858604 32483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:37:20.858700 32483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:37:20.858834 32483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:37:20.861389 32483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } }
I20260806 18:37:20.861778 32483 leader_election.cc:302] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f5fbd1ffd0a405ca92d6cbb14b4d088; no voters:
I20260806 18:37:20.862131 32483 leader_election.cc:288] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:37:20.862289 32487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:37:20.862566 32487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [term 2 LEADER]: Becoming Leader. State: Replica: 9f5fbd1ffd0a405ca92d6cbb14b4d088, State: Running, Role: LEADER
I20260806 18:37:20.862983 32487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } }
I20260806 18:37:20.863461 32483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:20.864073 32487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f5fbd1ffd0a405ca92d6cbb14b4d088. Latest consensus state: current_term: 2 leader_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } } }
I20260806 18:37:20.864315 32487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:20.864555 32489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5fbd1ffd0a405ca92d6cbb14b4d088" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34319 } } }
I20260806 18:37:20.864679 32489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:20.864938 32495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:37:20.868512 32495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:37:20.869143 32483 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:37:20.869261 32495 catalog_manager.cc:1259] Loaded cluster ID: d891f43e9466464e847278eafc66cb55
I20260806 18:37:20.869310 32495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:37:20.871750 32495 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:37:20.872462 32495 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088: Loaded TSK: 0
I20260806 18:37:20.876574 32495 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:37:20.960366 32485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:20.960687 32485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:20.960820 32485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:20.966109 32485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:20.966214 32485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:20.966329 32485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:37:20.973062 32485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.125.148:54461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:37465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=40325
--tserver_master_addrs=127.18.125.190:34319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:20.974485 32485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:20.975756 32485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:20.989615 32522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:20.989771 32521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:20.989621 32524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:20.991241 32485 server_base.cc:1034] running on GCE node
I20260806 18:37:20.991878 32485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:20.992857 32485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:20.994071 32485 hybrid_clock.cc:648] HybridClock initialized: now 1786041440994038 us; error 45 us; skew 500 ppm
Aug 06 18:37:20 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:20.997148 32485 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:37:20.998250 32485 webserver.cc:460] Webserver started at http://127.18.125.129:40325/ using document root <none> and password file <none>
I20260806 18:37:20.998845 32485 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:20.998957 32485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:37:21 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:37:21.027192 32485 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 18:37:21.032109 32533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:21.033922 32485 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260806 18:37:21.034070 32485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "3c5c00de630345cfa531883548165618"
format_stamp: "Formatted at 2026-08-06 18:37:19 on dist-test-slave-8w9v"
server_key: "a04a18f0082eef9f004f14f589377b2c"
server_key_iv: "e83797773af327b6b09fd3e41c0136d7"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:21.034832 32485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:21.066143 32485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:21.070150 32485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:21.070374 32485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:21.071079 32485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:21.072309 32485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:21.072373 32485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:21.072428 32485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:21.072490 32485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:21.084206 32485 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:37465
I20260806 18:37:21.086054 32485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:37:21.087065 32650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:37465 every 8 connection(s)
I20260806 18:37:21.096330 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 32485
I20260806 18:37:21.096961 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:54461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:39825
--local_ip_for_outbound_sockets=127.18.125.130
--tserver_master_addrs=127.18.125.190:34319
--webserver_port=35207
--webserver_interface=127.18.125.130
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.125.148:54461/kms with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:37:21 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:21.111711 32654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:21.091758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:38321 (local address 127.18.125.190:34319)
0806 18:37:21.092392 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:21.092414 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:21.092463 (+ 49us) server_negotiation.cc:407] Connection header received
0806 18:37:21.093430 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:21.093445 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:21.093858 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:21.094219 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:21.096172 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.097409 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.099126 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.099873 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.100102 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:21.104876 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:21.104885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:21.104906 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:21.104949 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:21.107506 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:21.108342 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:21.108350 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:21.108356 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:21.108534 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:21.109009 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:21.109013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:21.109015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:21.109614 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:21.109764 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:21.110416 (+ 652us) server_negotiation.cc:299] Negotiation successful
0806 18:37:21.110689 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":124,"threads_started":1}
I20260806 18:37:21.111725 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:21.090306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34319 (local address 127.18.125.129:38321)
0806 18:37:21.091885 (+ 1579us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:21.091946 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:21.092956 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:21.094485 (+ 1529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:21.094494 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:21.094864 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:21.095911 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:21.095936 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.097554 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.097558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:21.098962 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:21.098975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.100097 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.100101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:21.100311 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:21.102582 (+ 2271us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:21.102603 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:21.104545 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:21.107690 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:21.107698 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:21.107716 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:21.108160 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:21.108662 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:21.108666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:21.108668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:21.108878 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:21.109751 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:21.109758 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:21.110062 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 18:37:21.110265 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 18:37:21.110651 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1324,"thread_start_us":167,"threads_started":1}
I20260806 18:37:21.115202 32651 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34319
I20260806 18:37:21.115466 32651 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:21.116093 32651 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
W20260806 18:37:21.118886 32651 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:34319 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:37:21.119036 32651 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:21.119251 32651 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
I20260806 18:37:21.120491 32651 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:21.120671 32651 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
W20260806 18:37:21.121562 32651 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:37:21.268187 32655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:21.268582 32655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:21.268698 32655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:21.273365 32655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:21.273459 32655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:21.273555 32655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:37:21.280388 32655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.125.148:54461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:39825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=35207
--tserver_master_addrs=127.18.125.190:34319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:21.281692 32655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:21.282902 32655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:21.294453 32660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:21.294440 32661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:21.294664 32663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:21.295372 32655 server_base.cc:1034] running on GCE node
I20260806 18:37:21.295775 32655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:21.296613 32655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:21.297865 32655 hybrid_clock.cc:648] HybridClock initialized: now 1786041441297841 us; error 30 us; skew 500 ppm
Aug 06 18:37:21 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:21.301779 32655 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:37:21.303087 32655 webserver.cc:460] Webserver started at http://127.18.125.130:35207/ using document root <none> and password file <none>
I20260806 18:37:21.303956 32655 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:21.304097 32655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:37:21 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:37:21.327353 32655 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:37:21.334177 32672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:21.336061 32655 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260806 18:37:21.336233 32655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "29c7d42bc9714223add03117ddb3e59a"
format_stamp: "Formatted at 2026-08-06 18:37:19 on dist-test-slave-8w9v"
server_key: "eb939d085a505e1609cb676d857c4a32"
server_key_iv: "cf3bfbc29016d831009e99ee38860a7c"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:21.337044 32655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:21.382134 32655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:21.386307 32655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:21.386587 32655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:21.387346 32655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:21.388523 32655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:21.388631 32655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:21.388733 32655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:21.388798 32655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:21.403613 32655 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:39825
I20260806 18:37:21.405113 32655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:37:21.405098 324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:39825 every 8 connection(s)
I20260806 18:37:21.411665 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 32655
I20260806 18:37:21.412313 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.125.148:54461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:44185
--local_ip_for_outbound_sockets=127.18.125.131
--tserver_master_addrs=127.18.125.190:34319
--webserver_port=42011
--webserver_interface=127.18.125.131
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.125.148:54461/kms with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:37:21 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:21.425487 32654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:21.407956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:50907 (local address 127.18.125.190:34319)
0806 18:37:21.408179 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:21.408185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:21.409339 (+ 1154us) server_negotiation.cc:407] Connection header received
0806 18:37:21.410517 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:21.410522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:21.410601 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:21.410732 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:21.412536 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.413439 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.415162 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.415608 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.415749 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:21.419634 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:21.419641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:21.419647 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:21.419691 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:21.421992 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:21.422878 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:21.422886 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:21.422890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:21.423003 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:21.423560 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:21.423567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:21.423571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:21.423879 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:21.423994 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:21.425015 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0806 18:37:21.425221 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 18:37:21.427923 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:21.408523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34319 (local address 127.18.125.130:50907)
0806 18:37:21.409126 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:21.409188 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:21.410245 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:21.410978 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:21.410989 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:21.411441 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:21.412283 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:21.412310 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.413662 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.413669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:21.414994 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:21.415007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.416020 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.416028 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:21.416344 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:21.417195 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:21.417238 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:21.419338 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:21.422187 (+ 2849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:21.422199 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:21.422237 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:21.422714 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:21.423153 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:21.423158 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:21.423161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:21.423411 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:21.424089 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:21.424098 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:21.424575 (+ 477us) client_negotiation.cc:769] Sending connection context
0806 18:37:21.426135 (+ 1560us) client_negotiation.cc:240] Negotiation successful
0806 18:37:21.426609 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":200,"threads_started":1}
I20260806 18:37:21.429750 325 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34319
I20260806 18:37:21.430035 325 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:21.430720 325 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
W20260806 18:37:21.432751 325 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:34319 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:37:21.432904 325 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:21.433117 325 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
I20260806 18:37:21.434203 325 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:21.434448 325 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
W20260806 18:37:21.435447 325 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:37:21.565336 328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:21.565682 328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:21.565760 328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:21.571700 328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:21.571825 328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:21.571935 328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:37:21.578091 328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.125.148:54461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:44185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=42011
--tserver_master_addrs=127.18.125.190:34319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:21.579324 328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:21.580535 328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:21.592772 335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:21.592790 334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:21.593376 328 server_base.cc:1034] running on GCE node
W20260806 18:37:21.593730 337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:21.594415 328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:21.595310 328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:21.596583 328 hybrid_clock.cc:648] HybridClock initialized: now 1786041441596557 us; error 46 us; skew 500 ppm
Aug 06 18:37:21 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:21.603057 328 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:37:21.604590 328 webserver.cc:460] Webserver started at http://127.18.125.131:42011/ using document root <none> and password file <none>
I20260806 18:37:21.605515 328 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:21.605715 328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:37:21 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:37:21.633186 328 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:37:21.636977 345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:21.639362 328 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260806 18:37:21.639532 328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "44ff315422f24904805a37566201ed16"
format_stamp: "Formatted at 2026-08-06 18:37:20 on dist-test-slave-8w9v"
server_key: "70363c52ba10745b13736ca7adb9d587"
server_key_iv: "0a7cd3eef99f23967307e67dd3644506"
server_key_version: "kuduclusterkey@0"
I20260806 18:37:21.640406 328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:21.663988 328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:21.667552 328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:21.667780 328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:21.668553 328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:21.669452 328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:21.669528 328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:21.669602 328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:21.669646 328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:21.685132 328 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:44185
I20260806 18:37:21.685235 464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:44185 every 8 connection(s)
I20260806 18:37:21.686937 328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:37:21.696722 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 328
Aug 06 18:37:21 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:21.711872 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:21.697543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.18.125.190:34319)
0806 18:37:21.697912 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:21.697917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:21.698196 (+ 279us) server_negotiation.cc:407] Connection header received
0806 18:37:21.698449 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:21.698452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:21.698516 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:21.698640 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:21.699602 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.700440 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.703055 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.703395 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.703564 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:21.706994 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:21.706998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:21.707001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:21.707034 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:21.709809 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:21.710321 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:21.710325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:21.710328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:21.710428 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:21.710811 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:21.710815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:21.710817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:21.711070 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:21.711324 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:21.711529 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 18:37:21.711680 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":108,"threads_started":1}
I20260806 18:37:21.713007 32654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:21.691718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:60633 (local address 127.18.125.190:34319)
0806 18:37:21.691881 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:21.691886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:21.692983 (+ 1097us) server_negotiation.cc:407] Connection header received
0806 18:37:21.694280 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:21.694284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:21.694350 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:21.694434 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:21.695683 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.696376 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.701943 (+ 5567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.702305 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.702434 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:21.707579 (+ 5145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:21.707583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:21.707587 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:21.707643 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:21.709394 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:21.710909 (+ 1515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:21.710912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:21.710914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:21.711005 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:21.711538 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:21.711542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:21.711543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:21.711766 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:21.711854 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:21.712725 (+ 871us) server_negotiation.cc:299] Negotiation successful
0806 18:37:21.712846 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 18:37:21.714485 467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:21.692281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34319 (local address 127.18.125.131:60633)
0806 18:37:21.692808 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:21.692865 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:21.694044 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:21.694646 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:21.694656 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:21.695014 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:21.695526 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:21.695539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.700272 (+ 4733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.700276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:21.701783 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:21.701797 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:21.702633 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:21.702638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:21.702866 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:21.704367 (+ 1501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:21.704391 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:21.707346 (+ 2955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:21.710333 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:21.710339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:21.710355 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:21.710787 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:21.711110 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:21.711113 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:21.711115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:21.711416 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:21.712065 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:21.712073 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:21.712424 (+ 351us) client_negotiation.cc:769] Sending connection context
0806 18:37:21.713242 (+ 818us) client_negotiation.cc:240] Negotiation successful
0806 18:37:21.713580 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":151,"threads_started":1}
I20260806 18:37:21.716022 465 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34319
I20260806 18:37:21.716419 465 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:21.717047 465 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
W20260806 18:37:21.718852 465 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:34319 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:37:21.719012 465 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:21.719228 465 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
I20260806 18:37:21.720188 465 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:21.720429 465 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
W20260806 18:37:21.721292 465 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 18:37:22 dist-test-slave-8w9v krb5kdc[31403](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:22.121838 32651 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:22.122205 32651 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
I20260806 18:37:22.123724 32445 ts_manager.cc:194] Registered new tserver with Master: 3c5c00de630345cfa531883548165618 (127.18.125.129:37465)
I20260806 18:37:22.126080 32445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:38321
I20260806 18:37:22.435756 325 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:22.436198 325 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
I20260806 18:37:22.437364 32445 ts_manager.cc:194] Registered new tserver with Master: 29c7d42bc9714223add03117ddb3e59a (127.18.125.130:39825)
I20260806 18:37:22.438166 32445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:50907
Aug 06 18:37:22 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:37:22.721606 465 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:22.721966 465 heartbeater.cc:511] Master 127.18.125.190:34319 requested a full tablet report, sending...
I20260806 18:37:22.723023 32445 ts_manager.cc:194] Registered new tserver with Master: 44ff315422f24904805a37566201ed16 (127.18.125.131:44185)
I20260806 18:37:22.723951 32445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:60633
I20260806 18:37:22.728036 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:37:22.771391 486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:22.736826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.18.125.190:34319)
0806 18:37:22.745967 (+ 9141us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:22.745974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:22.748800 (+ 2826us) server_negotiation.cc:407] Connection header received
0806 18:37:22.748995 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:22.749001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:22.749083 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:22.749223 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:22.750587 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.751445 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.752748 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.753256 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.753405 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:22.765763 (+ 12358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:22.765770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:22.765775 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:22.765831 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:22.768063 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:22.768798 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:22.768802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:22.768804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:22.768924 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:22.769355 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:22.769360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:22.769364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:22.769670 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:22.769839 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:22.770909 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0806 18:37:22.771104 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9000,"thread_start_us":105,"threads_started":1}
I20260806 18:37:22.780300 32445 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:37:22.791883 32445 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:37:22.844364 498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:22.829862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:39825 (local address 127.0.0.1:58858)
0806 18:37:22.834858 (+ 4996us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:22.834876 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:22.834981 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:22.837354 (+ 2373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:22.837362 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:22.837394 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:22.837583 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:22.837593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.839154 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.839157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:22.842029 (+ 2872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:22.842043 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.843573 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.843577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:22.843796 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:22.843856 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 18:37:22.843985 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 18:37:22.844160 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4927,"thread_start_us":524,"threads_started":1}
I20260806 18:37:22.846216 497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:22.831170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58858 (local address 127.18.125.130:39825)
0806 18:37:22.836584 (+ 5414us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:22.836592 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:22.836626 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:37:22.836744 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:22.836751 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:22.837055 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:22.837316 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:22.837785 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.838995 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.842205 (+ 3210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.843173 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.843333 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:22.843493 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:22.845899 (+ 2406us) server_negotiation.cc:299] Negotiation successful
0806 18:37:22.846037 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5248,"thread_start_us":107,"threads_started":1}
I20260806 18:37:22.850281 32703 tablet_service.cc:1467] Processing CreateTablet for tablet ad90f6beb26a4bb1b9d6e5d5c7829128 (DEFAULT_TABLE table=test-table [id=fde907d2b78144ffa0e79fd1687eaebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:22.852037 32703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad90f6beb26a4bb1b9d6e5d5c7829128. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 18:37:22 dist-test-slave-8w9v krb5kdc[31403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for HTTP/127.18.125.148@KRBTEST.COM
I20260806 18:37:22.858053 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:22.827552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:37465 (local address 127.0.0.1:55200)
0806 18:37:22.832463 (+ 4911us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:22.832533 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:22.832680 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:22.839216 (+ 6536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:22.839219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:22.839239 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:22.839351 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:22.839361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.854504 (+ 15143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.854509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:22.855688 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:22.855708 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.857509 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.857514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:22.857622 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:22.857655 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 18:37:22.857775 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 18:37:22.857876 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4683,"thread_start_us":125,"threads_started":1}
I20260806 18:37:22.861532 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:22.833693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:44185 (local address 127.0.0.1:41340)
0806 18:37:22.837680 (+ 3987us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:22.837699 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:22.837821 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:22.839406 (+ 1585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:22.839411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:22.839435 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:22.839527 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:22.839534 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.841065 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.841069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:22.856299 (+ 15230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:22.856310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.861048 (+ 4738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.861052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:22.861163 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:22.861188 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:37:22.861267 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:37:22.861365 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3882,"thread_start_us":110,"threads_started":1}
I20260806 18:37:22.862389 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:22.834185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41340 (local address 127.18.125.131:44185)
0806 18:37:22.836353 (+ 2168us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:22.836375 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:22.837934 (+ 1559us) server_negotiation.cc:407] Connection header received
0806 18:37:22.838026 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:22.838032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:22.838233 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:22.838378 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:22.839726 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.840899 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.856766 (+ 15867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.860672 (+ 3906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.860826 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:22.860966 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:22.862104 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0806 18:37:22.862232 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":954,"thread_start_us":141,"threads_started":1}
I20260806 18:37:22.865213 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:22.828820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55200 (local address 127.18.125.129:37465)
0806 18:37:22.838751 (+ 9931us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:22.838760 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:22.838786 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:37:22.838869 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:22.838874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:22.839059 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:22.839220 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:22.852211 (+ 12991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.854335 (+ 2124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.856471 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:22.857333 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:22.857504 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:22.857677 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:22.864916 (+ 7239us) server_negotiation.cc:299] Negotiation successful
0806 18:37:22.865064 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9780,"thread_start_us":85,"threads_started":1}
I20260806 18:37:22.866716 384 tablet_service.cc:1467] Processing CreateTablet for tablet ad90f6beb26a4bb1b9d6e5d5c7829128 (DEFAULT_TABLE table=test-table [id=fde907d2b78144ffa0e79fd1687eaebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:22.868018 384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad90f6beb26a4bb1b9d6e5d5c7829128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:22.868383 32572 tablet_service.cc:1467] Processing CreateTablet for tablet ad90f6beb26a4bb1b9d6e5d5c7829128 (DEFAULT_TABLE table=test-table [id=fde907d2b78144ffa0e79fd1687eaebd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:22.869524 32572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad90f6beb26a4bb1b9d6e5d5c7829128. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:22.881073 500 tablet_bootstrap.cc:492] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618: Bootstrap starting.
I20260806 18:37:22.883797 500 tablet_bootstrap.cc:654] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:22.885123 500 log.cc:824] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:22.893075 502 tablet_bootstrap.cc:492] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a: Bootstrap starting.
I20260806 18:37:22.896102 502 tablet_bootstrap.cc:654] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:22.897158 502 log.cc:824] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:22.899611 502 tablet_bootstrap.cc:492] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a: No bootstrap required, opened a new log
I20260806 18:37:22.899847 502 ts_tablet_manager.cc:1397] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260806 18:37:22.904670 504 tablet_bootstrap.cc:492] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: Bootstrap starting.
I20260806 18:37:22.905902 500 tablet_bootstrap.cc:492] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618: No bootstrap required, opened a new log
I20260806 18:37:22.906167 500 ts_tablet_manager.cc:1397] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.000s
I20260806 18:37:22.911842 500 raft_consensus.cc:348] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c5c00de630345cfa531883548165618" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37465 } } peers { permanent_uuid: "29c7d42bc9714223add03117ddb3e59a" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39825 } } peers { permanent_uuid: "44ff315422f24904805a37566201ed16" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44185 } }
I20260806 18:37:22.912218 500 raft_consensus.cc:374] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:22.912284 500 raft_consensus.cc:729] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c5c00de630345cfa531883548165618, State: Initialized, Role: FOLLOWER
I20260806 18:37:22.912650 502 raft_consensus.cc:348] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c5c00de630345cfa531883548165618" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37465 } } peers { permanent_uuid: "29c7d42bc9714223add03117ddb3e59a" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39825 } } peers { permanent_uuid: "44ff315422f24904805a37566201ed16" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44185 } }
I20260806 18:37:22.912961 502 raft_consensus.cc:374] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:22.913026 502 raft_consensus.cc:729] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c7d42bc9714223add03117ddb3e59a, State: Initialized, Role: FOLLOWER
I20260806 18:37:22.913565 502 consensus_queue.cc:260] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c5c00de630345cfa531883548165618" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37465 } } peers { permanent_uuid: "29c7d42bc9714223add03117ddb3e59a" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39825 } } peers { permanent_uuid: "44ff315422f24904805a37566201ed16" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44185 } }
I20260806 18:37:22.914856 502 ts_tablet_manager.cc:1428] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a: Time spent starting tablet: real 0.015s user 0.002s sys 0.005s
I20260806 18:37:22.914732 504 tablet_bootstrap.cc:654] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:22.915409 500 consensus_queue.cc:260] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c5c00de630345cfa531883548165618" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37465 } } peers { permanent_uuid: "29c7d42bc9714223add03117ddb3e59a" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39825 } } peers { permanent_uuid: "44ff315422f24904805a37566201ed16" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44185 } }
I20260806 18:37:22.916641 500 ts_tablet_manager.cc:1428] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260806 18:37:22.917058 32651 heartbeater.cc:503] Master 127.18.125.190:34319 was elected leader, sending a full tablet report...
I20260806 18:37:22.918593 504 log.cc:824] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:22.921326 325 heartbeater.cc:503] Master 127.18.125.190:34319 was elected leader, sending a full tablet report...
I20260806 18:37:22.929840 504 tablet_bootstrap.cc:492] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: No bootstrap required, opened a new log
I20260806 18:37:22.930677 504 ts_tablet_manager.cc:1397] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.000s
I20260806 18:37:22.935802 504 raft_consensus.cc:348] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c5c00de630345cfa531883548165618" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37465 } } peers { permanent_uuid: "29c7d42bc9714223add03117ddb3e59a" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39825 } } peers { permanent_uuid: "44ff315422f24904805a37566201ed16" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44185 } }
I20260806 18:37:22.937322 504 raft_consensus.cc:374] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:22.937747 504 raft_consensus.cc:729] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ff315422f24904805a37566201ed16, State: Initialized, Role: FOLLOWER
I20260806 18:37:22.939110 504 consensus_queue.cc:260] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c5c00de630345cfa531883548165618" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37465 } } peers { permanent_uuid: "29c7d42bc9714223add03117ddb3e59a" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39825 } } peers { permanent_uuid: "44ff315422f24904805a37566201ed16" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44185 } }
I20260806 18:37:22.940805 465 heartbeater.cc:503] Master 127.18.125.190:34319 was elected leader, sending a full tablet report...
I20260806 18:37:22.942600 504 ts_tablet_manager.cc:1428] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
W20260806 18:37:22.943118 466 tablet.cc:2369] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:37:23.030651 508 raft_consensus.cc:482] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:23.030917 508 raft_consensus.cc:504] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c5c00de630345cfa531883548165618" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37465 } } peers { permanent_uuid: "29c7d42bc9714223add03117ddb3e59a" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39825 } } peers { permanent_uuid: "44ff315422f24904805a37566201ed16" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44185 } }
I20260806 18:37:23.032537 508 leader_election.cc:288] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c5c00de630345cfa531883548165618 (127.18.125.129:37465), 29c7d42bc9714223add03117ddb3e59a (127.18.125.130:39825)
I20260806 18:37:23.047616 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:23.034194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:34187 (local address 127.18.125.129:37465)
0806 18:37:23.034370 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:23.034375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:23.037375 (+ 3000us) server_negotiation.cc:407] Connection header received
0806 18:37:23.038132 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:23.038136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:23.038193 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:23.038249 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:23.038768 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:23.039552 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:23.046073 (+ 6521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:23.046731 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:23.046847 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:23.046937 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:23.047370 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 18:37:23.047465 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:37:23.048403 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:23.033199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:37465 (local address 127.18.125.131:34187)
0806 18:37:23.037256 (+ 4057us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:23.037276 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:23.037947 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:23.038421 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:23.038424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:23.038445 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:23.038609 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:23.038617 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:23.040477 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:23.040481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:23.044519 (+ 4038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:23.044532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:23.047057 (+ 2525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:23.047061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:23.047188 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:23.047215 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:37:23.048105 (+ 890us) client_negotiation.cc:240] Negotiation successful
0806 18:37:23.048220 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3945,"thread_start_us":123,"threads_started":1}
I20260806 18:37:23.049321 32584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad90f6beb26a4bb1b9d6e5d5c7829128" candidate_uuid: "44ff315422f24904805a37566201ed16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c5c00de630345cfa531883548165618" is_pre_election: true
I20260806 18:37:23.049710 32584 raft_consensus.cc:2454] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44ff315422f24904805a37566201ed16 in term 0.
I20260806 18:37:23.050460 497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:23.034373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:40511 (local address 127.18.125.130:39825)
0806 18:37:23.034527 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:23.034532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:23.042846 (+ 8314us) server_negotiation.cc:407] Connection header received
0806 18:37:23.044216 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:23.044220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:23.044282 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:23.044386 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:23.046724 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:23.047516 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:23.049117 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:23.049719 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:23.049823 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:23.049874 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:23.050248 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 18:37:23.050338 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:37:23.050446 349 leader_election.cc:302] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c5c00de630345cfa531883548165618, 44ff315422f24904805a37566201ed16; no voters:
I20260806 18:37:23.050863 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:23.033645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:39825 (local address 127.18.125.131:40511)
0806 18:37:23.042722 (+ 9077us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:23.042741 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:23.044037 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:23.044525 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:23.044529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:23.044548 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:23.044676 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:23.044682 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:23.047677 (+ 2995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:23.047680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:23.048958 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:23.048970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:23.049994 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:23.049998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:23.050097 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:23.050119 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:37:23.050630 (+ 511us) client_negotiation.cc:240] Negotiation successful
0806 18:37:23.050718 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8985,"thread_start_us":101,"threads_started":1}
I20260806 18:37:23.050872 508 raft_consensus.cc:2781] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:37:23.051134 508 raft_consensus.cc:482] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:37:23.051182 508 raft_consensus.cc:3037] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:23.052702 508 raft_consensus.cc:504] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c5c00de630345cfa531883548165618" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37465 } } peers { permanent_uuid: "29c7d42bc9714223add03117ddb3e59a" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39825 } } peers { permanent_uuid: "44ff315422f24904805a37566201ed16" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44185 } }
I20260806 18:37:23.053126 508 leader_election.cc:288] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [CANDIDATE]: Term 1 election: Requested vote from peers 3c5c00de630345cfa531883548165618 (127.18.125.129:37465), 29c7d42bc9714223add03117ddb3e59a (127.18.125.130:39825)
I20260806 18:37:23.054340 32736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad90f6beb26a4bb1b9d6e5d5c7829128" candidate_uuid: "44ff315422f24904805a37566201ed16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7d42bc9714223add03117ddb3e59a" is_pre_election: true
I20260806 18:37:23.054687 32736 raft_consensus.cc:2454] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44ff315422f24904805a37566201ed16 in term 0.
I20260806 18:37:23.055043 32743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad90f6beb26a4bb1b9d6e5d5c7829128" candidate_uuid: "44ff315422f24904805a37566201ed16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29c7d42bc9714223add03117ddb3e59a"
I20260806 18:37:23.055154 32743 raft_consensus.cc:3037] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:23.056620 32743 raft_consensus.cc:2454] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44ff315422f24904805a37566201ed16 in term 1.
I20260806 18:37:23.057168 349 leader_election.cc:302] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29c7d42bc9714223add03117ddb3e59a, 44ff315422f24904805a37566201ed16; no voters:
I20260806 18:37:23.057391 508 raft_consensus.cc:2781] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:23.057461 32584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad90f6beb26a4bb1b9d6e5d5c7829128" candidate_uuid: "44ff315422f24904805a37566201ed16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c5c00de630345cfa531883548165618"
I20260806 18:37:23.057608 32584 raft_consensus.cc:3037] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:23.057648 508 raft_consensus.cc:686] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 1 LEADER]: Becoming Leader. State: Replica: 44ff315422f24904805a37566201ed16, State: Running, Role: LEADER
I20260806 18:37:23.058046 508 consensus_queue.cc:237] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c5c00de630345cfa531883548165618" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37465 } } peers { permanent_uuid: "29c7d42bc9714223add03117ddb3e59a" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39825 } } peers { permanent_uuid: "44ff315422f24904805a37566201ed16" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44185 } }
I20260806 18:37:23.059041 32584 raft_consensus.cc:2454] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44ff315422f24904805a37566201ed16 in term 1.
I20260806 18:37:23.061839 32445 catalog_manager.cc:5495] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44ff315422f24904805a37566201ed16 (127.18.125.131). New cstate: current_term: 1 leader_uuid: "44ff315422f24904805a37566201ed16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c5c00de630345cfa531883548165618" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29c7d42bc9714223add03117ddb3e59a" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44ff315422f24904805a37566201ed16" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 44185 } health_report { overall_health: HEALTHY } } }
I20260806 18:37:23.078819 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:23.073473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41344 (local address 127.18.125.131:44185)
0806 18:37:23.073604 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:23.073609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:23.073622 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:37:23.073719 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:23.073722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:23.073799 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:23.073920 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:37:23.074342 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:23.075114 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:23.076414 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:23.076768 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:23.076900 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:23.077274 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:37:23.077955 (+ 681us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:37:23.078079 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:23.078564 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 18:37:23.078637 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:37:23.087098 32743 raft_consensus.cc:1261] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a [term 1 FOLLOWER]: Refusing update from remote peer 44ff315422f24904805a37566201ed16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:23.087100 32584 raft_consensus.cc:1261] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618 [term 1 FOLLOWER]: Refusing update from remote peer 44ff315422f24904805a37566201ed16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:23.087982 508 consensus_queue.cc:1035] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c5c00de630345cfa531883548165618" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 37465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:23.088498 508 consensus_queue.cc:1035] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29c7d42bc9714223add03117ddb3e59a" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:37:23.090315 32652 tablet.cc:2369] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:37:23.103188 515 mvcc.cc:205] Tried to move back new op lower bound from 7315625750873878528 to 7315625750769311744. Current Snapshot: MvccSnapshot[applied={T|T < 7315625750873878528}]
I20260806 18:37:23.105891 518 mvcc.cc:205] Tried to move back new op lower bound from 7315625750873878528 to 7315625750769311744. Current Snapshot: MvccSnapshot[applied={T|T < 7315625750873878528}]
I20260806 18:37:23.111086 516 mvcc.cc:205] Tried to move back new op lower bound from 7315625750873878528 to 7315625750769311744. Current Snapshot: MvccSnapshot[applied={T|T < 7315625750873878528}]
I20260806 18:37:23.123961 32444 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:23.124225 32444 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:23.127501 32444 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f5fbd1ffd0a405ca92d6cbb14b4d088: Sending DeleteTablet for 3 replicas of tablet ad90f6beb26a4bb1b9d6e5d5c7829128
I20260806 18:37:23.129034 384 tablet_service.cc:1514] Processing DeleteTablet for tablet ad90f6beb26a4bb1b9d6e5d5c7829128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:23 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:41340
I20260806 18:37:23.130370 32572 tablet_service.cc:1514] Processing DeleteTablet for tablet ad90f6beb26a4bb1b9d6e5d5c7829128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:23 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:55200
I20260806 18:37:23.130975 32703 tablet_service.cc:1514] Processing DeleteTablet for tablet ad90f6beb26a4bb1b9d6e5d5c7829128 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:23 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:58858
I20260806 18:37:23.132030 525 tablet_replica.cc:330] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 3c5c00de630345cfa531883548165618: stopping tablet replica
I20260806 18:37:23.132795 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 32485
I20260806 18:37:23.150734 526 tablet_replica.cc:330] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a: stopping tablet replica
I20260806 18:37:23.151211 526 raft_consensus.cc:2229] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:37:23.151607 526 raft_consensus.cc:2258] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:23.158751 524 tablet_replica.cc:330] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: stopping tablet replica
I20260806 18:37:23.159193 524 raft_consensus.cc:2229] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:37:23.159703 524 raft_consensus.cc:2258] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:23.161695 526 ts_tablet_manager.cc:1905] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:23.165518 526 ts_tablet_manager.cc:1918] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:23.165656 526 log.cc:1196] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/ad90f6beb26a4bb1b9d6e5d5c7829128
I20260806 18:37:23.165719 524 ts_tablet_manager.cc:1905] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:23.166038 526 ts_tablet_manager.cc:1939] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 29c7d42bc9714223add03117ddb3e59a: Deleting consensus metadata
I20260806 18:37:23.167642 32432 catalog_manager.cc:4841] TS 29c7d42bc9714223add03117ddb3e59a (127.18.125.130:39825): tablet ad90f6beb26a4bb1b9d6e5d5c7829128 (table test-table [id=fde907d2b78144ffa0e79fd1687eaebd]) successfully deleted
W20260806 18:37:23.168218 32432 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:37465 (error 108)
I20260806 18:37:23.169230 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 32655
I20260806 18:37:23.178213 524 ts_tablet_manager.cc:1918] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:23.178380 524 log.cc:1196] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/ad90f6beb26a4bb1b9d6e5d5c7829128
I20260806 18:37:23.178788 524 ts_tablet_manager.cc:1939] T ad90f6beb26a4bb1b9d6e5d5c7829128 P 44ff315422f24904805a37566201ed16: Deleting consensus metadata
I20260806 18:37:23.180450 32430 catalog_manager.cc:4841] TS 44ff315422f24904805a37566201ed16 (127.18.125.131:44185): tablet ad90f6beb26a4bb1b9d6e5d5c7829128 (table test-table [id=fde907d2b78144ffa0e79fd1687eaebd]) successfully deleted
I20260806 18:37:23.181902 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:23.174787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:37465 (local address 127.0.0.1:55216)
0806 18:37:23.181688 (+ 6901us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:23.181799 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:37465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6786}
W20260806 18:37:23.182160 32432 catalog_manager.cc:4584] TS 3c5c00de630345cfa531883548165618 (127.18.125.129:37465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad90f6beb26a4bb1b9d6e5d5c7829128: Network error: Client connection negotiation failed: client connection to 127.18.125.129:37465: connect: Connection refused (error 111)
I20260806 18:37:23.190900 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 328
I20260806 18:37:23.201706 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 32412
I20260806 18:37:23.210592 18934 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 18:37:23.556845 18934 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 18:37:23.556978 18934 mini_ranger.cc:66] Stopping Ranger...
I20260806 18:37:23.903438 18934 mini_ranger.cc:68] Stopped Ranger
2026-08-06 18:37:23.903 UTC [31436] LOG: received smart shutdown request
2026-08-06 18:37:23.909 UTC [31436] LOG: background worker "logical replication launcher" (PID 31445) exited with exit code 1
2026-08-06 18:37:23.909 UTC [31440] LOG: shutting down
2026-08-06 18:37:24.098 UTC [31436] LOG: database system is shut down
2026-08-06T18:37:24Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55192 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:37:24 dist-test-slave-8w9v krb5kdc[539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:37:24 dist-test-slave-8w9v krb5kdc[539](info): set up 2 sockets
Aug 06 18:37:24 dist-test-slave-8w9v krb5kdc[539](info): commencing operation
krb5kdc: starting...
W20260806 18:37:25.314800 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:37:25 dist-test-slave-8w9v krb5kdc[539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:37:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:37:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:37:25.449015 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:36583
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:46377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:36583,127.18.125.189:38213,127.18.125.188:36307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:25.567837 552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:25.568243 552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:25.568387 552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:25.572556 552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:25.572665 552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:25.572705 552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:25.572744 552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:25.572780 552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:25.578583 552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:46377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:36583,127.18.125.189:38213,127.18.125.188:36307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:36583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:25.579802 552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:25.580909 552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:25.589772 557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:25.589772 558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:25.589869 560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:25.590201 552 server_base.cc:1034] running on GCE node
I20260806 18:37:25.590862 552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:25.592056 552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:25.593271 552 hybrid_clock.cc:648] HybridClock initialized: now 1786041445593256 us; error 25 us; skew 500 ppm
Aug 06 18:37:25 dist-test-slave-8w9v krb5kdc[539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:25.596376 552 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:37:25.597416 552 webserver.cc:460] Webserver started at http://127.18.125.190:35157/ using document root <none> and password file <none>
I20260806 18:37:25.598061 552 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:25.598160 552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:25.598384 552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:25.600632 552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "2c661ffdb52d40fd87132252fe507edd"
format_stamp: "Formatted at 2026-08-06 18:37:25 on dist-test-slave-8w9v"
server_key: "478c45b49ecec053a45d28b242cfda56"
server_key_iv: "05420fce21e88f642a429d324f04a44f"
server_key_version: "encryptionkey@0"
I20260806 18:37:25.601223 552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "2c661ffdb52d40fd87132252fe507edd"
format_stamp: "Formatted at 2026-08-06 18:37:25 on dist-test-slave-8w9v"
server_key: "478c45b49ecec053a45d28b242cfda56"
server_key_iv: "05420fce21e88f642a429d324f04a44f"
server_key_version: "encryptionkey@0"
I20260806 18:37:25.605239 552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:37:25.608021 567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:25.609733 552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:37:25.609939 552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "2c661ffdb52d40fd87132252fe507edd"
format_stamp: "Formatted at 2026-08-06 18:37:25 on dist-test-slave-8w9v"
server_key: "478c45b49ecec053a45d28b242cfda56"
server_key_iv: "05420fce21e88f642a429d324f04a44f"
server_key_version: "encryptionkey@0"
I20260806 18:37:25.610095 552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:25.627981 552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:25.631484 552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:25.631726 552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:25.639899 552 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:36583
I20260806 18:37:25.640383 621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:36583 every 8 connection(s)
I20260806 18:37:25.641080 552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:37:25.644331 622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:25.647004 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 552
I20260806 18:37:25.647161 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:37:25.647478 18934 external_mini_cluster.cc:1428] Setting key 6da66f9eb4e4ea798e77029868e5f07c
I20260806 18:37:25.648985 622 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.125.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.189@KRBTEST.COM" created.
Aug 06 18:37:25 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:25.666133 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.651515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33606 (local address 127.18.125.190:36583)
0806 18:37:25.652157 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:25.652179 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:25.652400 (+ 221us) server_negotiation.cc:407] Connection header received
0806 18:37:25.653519 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:25.653524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:25.653718 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:25.653926 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:25.654877 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.655833 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.656957 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.657356 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.657561 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:25.660303 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:25.660313 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:25.660333 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:25.660370 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:25.662340 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:25.663041 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:25.663047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:25.663048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:25.663289 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:25.663710 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:25.663714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:25.663715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:25.664245 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:25.664371 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:25.664948 (+ 577us) server_negotiation.cc:299] Negotiation successful
0806 18:37:25.665184 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":487,"thread_start_us":194,"threads_started":1}
I20260806 18:37:25.666139 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.651515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:36583 (local address 127.0.0.1:33606)
0806 18:37:25.652226 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.652278 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:25.653172 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:25.653926 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:25.653932 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:25.654435 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:25.654686 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:25.654702 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.656001 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.656004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:25.656818 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:25.656830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.657545 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.657550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:25.657678 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:25.658367 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:25.658387 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:25.660056 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:25.662480 (+ 2424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:25.662495 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:25.662510 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:25.662885 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:25.663415 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:25.663422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:25.663424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:25.663594 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:25.664391 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:25.664402 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:25.664603 (+ 201us) client_negotiation.cc:769] Sending connection context
0806 18:37:25.664826 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 18:37:25.665276 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":64,"spinlock_wait_cycles":16768,"thread_start_us":182,"threads_started":1}
I20260806 18:37:25.669034 622 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:37:25.670083 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.669671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:38213 (local address 127.0.0.1:39036)
0806 18:37:25.669828 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.669947 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:38213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 18:37:25.670392 570 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.125.189:38213: connect: Connection refused (error 111)
I20260806 18:37:25.671469 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.671229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:38213 (local address 127.0.0.1:39048)
0806 18:37:25.671363 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.671413 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:38213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 18:37:25.671743 622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.189:38213: Network error: Client connection negotiation failed: client connection to 127.18.125.189:38213: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab.
Entry for principal kudu/127.18.125.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab.
I20260806 18:37:25.677320 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.189:38213
--webserver_interface=127.18.125.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab
--principal=kudu/127.18.125.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:46377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:36583,127.18.125.189:38213,127.18.125.188:36307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:37:25.721002 622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } attempt: 1
I20260806 18:37:25.722002 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.721603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:38213 (local address 127.0.0.1:39062)
0806 18:37:25.721831 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.721906 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:38213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 18:37:25.722787 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.722551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:38213 (local address 127.0.0.1:39072)
0806 18:37:25.722670 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.722723 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:38213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 18:37:25.722951 622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.189:38213: Network error: Client connection negotiation failed: client connection to 127.18.125.189:38213: connect: Connection refused (error 111)
I20260806 18:37:25.791163 622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } attempt: 2
I20260806 18:37:25.792202 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.791767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:38213 (local address 127.0.0.1:39088)
0806 18:37:25.791968 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.792052 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:38213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 18:37:25.792975 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.792718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:38213 (local address 127.0.0.1:39104)
0806 18:37:25.792850 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.792909 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:38213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 18:37:25.793131 622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.189:38213: Network error: Client connection negotiation failed: client connection to 127.18.125.189:38213: connect: Connection refused (error 111)
W20260806 18:37:25.798331 629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:25.798611 629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:25.798710 629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:25.802884 629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:25.802956 629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:25.802979 629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:25.802999 629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:25.803058 629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:25.808786 629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:46377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:36583,127.18.125.189:38213,127.18.125.188:36307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab
--principal=kudu/127.18.125.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.189:38213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.125.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:25.809974 629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:25.811069 629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:25.822345 639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:25.822398 629 server_base.cc:1034] running on GCE node
W20260806 18:37:25.822275 636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:25.822278 634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:25.823731 629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:25.825067 629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:25.826241 629 hybrid_clock.cc:648] HybridClock initialized: now 1786041445826210 us; error 47 us; skew 500 ppm
Aug 06 18:37:25 dist-test-slave-8w9v krb5kdc[539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:25.829321 629 init.cc:377] Logged in from keytab as kudu/127.18.125.189@KRBTEST.COM (short username kudu)
I20260806 18:37:25.830333 629 webserver.cc:460] Webserver started at http://127.18.125.189:43423/ using document root <none> and password file <none>
I20260806 18:37:25.830947 629 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:25.831012 629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:25.831236 629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:25.833539 629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/data/instance:
uuid: "b8265cf13d014e64a59872bd395e3d58"
format_stamp: "Formatted at 2026-08-06 18:37:25 on dist-test-slave-8w9v"
server_key: "a8e838d4ea66d2d5ff5ad9bc399126fe"
server_key_iv: "327b25826401f060018ae477c5cf7bbe"
server_key_version: "encryptionkey@0"
I20260806 18:37:25.834578 629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/wal/instance:
uuid: "b8265cf13d014e64a59872bd395e3d58"
format_stamp: "Formatted at 2026-08-06 18:37:25 on dist-test-slave-8w9v"
server_key: "a8e838d4ea66d2d5ff5ad9bc399126fe"
server_key_iv: "327b25826401f060018ae477c5cf7bbe"
server_key_version: "encryptionkey@0"
I20260806 18:37:25.839210 629 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:37:25.842114 646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:25.843482 629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:37:25.843647 629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/wal
uuid: "b8265cf13d014e64a59872bd395e3d58"
format_stamp: "Formatted at 2026-08-06 18:37:25 on dist-test-slave-8w9v"
server_key: "a8e838d4ea66d2d5ff5ad9bc399126fe"
server_key_iv: "327b25826401f060018ae477c5cf7bbe"
server_key_version: "encryptionkey@0"
I20260806 18:37:25.843787 629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:25.867069 629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:25.870705 629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:25.870930 629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:25.878679 629 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.189:38213
I20260806 18:37:25.878980 698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.189:38213 every 8 connection(s)
I20260806 18:37:25.879784 629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
I20260806 18:37:25.882800 699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:25.884322 622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } attempt: 3
I20260806 18:37:25.885018 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 629
I20260806 18:37:25.885146 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-1/wal/instance
I20260806 18:37:25.885475 18934 external_mini_cluster.cc:1428] Setting key 82c212fec04cf8ffd570f39613bb0cd4
I20260806 18:37:25.888916 699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:37:25 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.188@KRBTEST.COM" created.
I20260806 18:37:25.898679 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.885005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:38213 (local address 127.0.0.1:39114)
0806 18:37:25.885179 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.885200 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:25.885314 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:25.887766 (+ 2452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:25.887770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:25.887985 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:25.888169 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:25.888179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.889720 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.889724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:25.890290 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:25.890301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.891372 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.891377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:25.891499 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:25.892188 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:25.892203 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:25.893923 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:25.896678 (+ 2755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:25.896682 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:25.896684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:25.896906 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:25.897295 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:25.897299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:25.897301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:25.897397 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:25.898057 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:25.898061 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:25.898167 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 18:37:25.898300 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 18:37:25.898484 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
Aug 06 18:37:25 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:25.901638 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.885649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.18.125.189:38213)
0806 18:37:25.886110 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:25.886132 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:25.886179 (+ 47us) server_negotiation.cc:407] Connection header received
0806 18:37:25.886914 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:25.886932 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:25.887385 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:25.887800 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:25.888329 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.889572 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.890432 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.891190 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.891823 (+ 633us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:25.894144 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:25.894155 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:25.894170 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:25.894213 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:25.896513 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:25.897031 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:25.897038 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:25.897045 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:25.897181 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:25.897496 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:25.897501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:25.897502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:25.897940 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:25.898093 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:25.898520 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 18:37:25.900407 (+ 1887us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":168,"threads_started":1}
I20260806 18:37:25.903817 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.891171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:36583 (local address 127.0.0.1:33620)
0806 18:37:25.892194 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.892252 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:25.892393 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:25.892781 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:25.892791 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:25.893170 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:25.893318 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:25.893324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.894215 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.894220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:25.895313 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:25.895328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.895995 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.895998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:25.896087 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:25.896680 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:25.896700 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:25.899419 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:25.901380 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:25.901389 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:25.901396 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:25.901635 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:25.902232 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:25.902235 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:25.902237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:25.902348 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:25.902964 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:25.902973 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:25.903123 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 18:37:25.903285 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 18:37:25.903514 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":132,"threads_started":1}
I20260806 18:37:25.904395 622 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:37:25.905426 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.905062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55778)
0806 18:37:25.905261 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.905337 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 18:37:25.905587 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.891245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.18.125.190:36583)
0806 18:37:25.891401 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:25.891406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:25.892354 (+ 948us) server_negotiation.cc:407] Connection header received
0806 18:37:25.892503 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:25.892509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:25.892579 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:25.892682 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:25.893457 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.894075 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.895468 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.895854 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.895981 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:25.899591 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:25.899599 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:25.899605 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:25.899638 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:25.901243 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:25.902023 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:25.902026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:25.902028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:25.902121 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:25.902457 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:25.902464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:25.902468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:25.902777 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:25.902901 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:25.905299 (+ 2398us) server_negotiation.cc:299] Negotiation successful
0806 18:37:25.905453 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:37:25.906335 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.906007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55788)
0806 18:37:25.906174 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.906234 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260806 18:37:25.906533 622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:36307: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
I20260806 18:37:25.907030 699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:37:25 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
I20260806 18:37:25.918444 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.907783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39128 (local address 127.18.125.189:38213)
0806 18:37:25.907936 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:25.907943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:25.907968 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:37:25.908027 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:25.908031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:25.908091 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:25.908266 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:25.908778 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.909736 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.910667 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.911135 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.911253 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:25.914625 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:25.914629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:25.914631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:25.914656 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:25.916559 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:25.917055 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:25.917058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:25.917060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:25.917195 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:25.917490 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:25.917493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:25.917494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:25.917718 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:25.917817 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:25.918143 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 18:37:25.918264 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 18:37:25.918450 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.907685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:38213 (local address 127.0.0.1:39128)
0806 18:37:25.907825 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.907841 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:25.907940 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:25.908304 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:25.908306 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:25.908488 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:25.908604 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:25.908610 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.909904 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.909907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:25.910509 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:25.910519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:25.911285 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:25.911289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:25.911406 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:25.912216 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:25.912236 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:25.914410 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:25.916690 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:25.916693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:25.916695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:25.916919 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:25.917296 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:25.917299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:25.917300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:25.917394 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:25.917841 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:25.917844 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:25.917942 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 18:37:25.918043 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 18:37:25.918176 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab.
Entry for principal kudu/127.18.125.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab.
I20260806 18:37:25.919756 699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:37:25.920323 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.188:36307
--webserver_interface=127.18.125.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab
--principal=kudu/127.18.125.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:46377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:36583,127.18.125.189:38213,127.18.125.188:36307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:37:25.920711 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.920305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55796)
0806 18:37:25.920538 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.920643 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260806 18:37:25.921521 650 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
I20260806 18:37:25.923854 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.923611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55802)
0806 18:37:25.923733 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.923781 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 18:37:25.924053 699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:36307: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
I20260806 18:37:25.937695 622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } attempt: 1
I20260806 18:37:25.938688 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.938288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55814)
0806 18:37:25.938490 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.938577 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 18:37:25.939551 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.939259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55818)
0806 18:37:25.939414 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.939479 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 18:37:25.939713 622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:36307: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
I20260806 18:37:25.973417 699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } attempt: 1
I20260806 18:37:25.974462 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.974117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55828)
0806 18:37:25.974307 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.974377 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 18:37:25.975261 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:25.975038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55838)
0806 18:37:25.975150 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:25.975204 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:37:25.975389 699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:36307: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
I20260806 18:37:26.014971 622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } attempt: 2
I20260806 18:37:26.015914 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.015542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55854)
0806 18:37:26.015729 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:26.015808 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 18:37:26.016855 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.016622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55862)
0806 18:37:26.016728 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:26.016787 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 18:37:26.016993 622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:36307: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
I20260806 18:37:26.043589 699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } attempt: 2
W20260806 18:37:26.043799 705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:26.044240 705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:26.044353 705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:37:26.044708 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.044368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55864)
0806 18:37:26.044543 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:26.044621 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 18:37:26.045600 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.045351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55872)
0806 18:37:26.045484 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:26.045542 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 18:37:26.045742 699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:36307: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
W20260806 18:37:26.049230 705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:26.049337 705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:26.049384 705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:26.049444 705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:26.049482 705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:26.055310 705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:46377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:36583,127.18.125.189:38213,127.18.125.188:36307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab
--principal=kudu/127.18.125.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.188:36307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.125.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:26.056648 705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:26.057852 705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:26.067482 714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:26.067448 711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:26.067428 712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:26.067603 705 server_base.cc:1034] running on GCE node
I20260806 18:37:26.068320 705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:26.070267 705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:26.071516 705 hybrid_clock.cc:648] HybridClock initialized: now 1786041446071502 us; error 38 us; skew 500 ppm
Aug 06 18:37:26 dist-test-slave-8w9v krb5kdc[539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041446, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:26.074730 705 init.cc:377] Logged in from keytab as kudu/127.18.125.188@KRBTEST.COM (short username kudu)
I20260806 18:37:26.075742 705 webserver.cc:460] Webserver started at http://127.18.125.188:41437/ using document root <none> and password file <none>
I20260806 18:37:26.076473 705 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:26.076547 705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:26.076740 705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:26.079028 705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/data/instance:
uuid: "92e09dcddb5b4847ba244f935e25a660"
format_stamp: "Formatted at 2026-08-06 18:37:26 on dist-test-slave-8w9v"
server_key: "567a1a5ac4b8ef7402e00d1c7c6acf39"
server_key_iv: "4aa82bdfe74a165d6c7801d19bfacb6c"
server_key_version: "encryptionkey@0"
I20260806 18:37:26.079653 705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/wal/instance:
uuid: "92e09dcddb5b4847ba244f935e25a660"
format_stamp: "Formatted at 2026-08-06 18:37:26 on dist-test-slave-8w9v"
server_key: "567a1a5ac4b8ef7402e00d1c7c6acf39"
server_key_iv: "4aa82bdfe74a165d6c7801d19bfacb6c"
server_key_version: "encryptionkey@0"
I20260806 18:37:26.083894 705 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:37:26.086640 721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:26.088040 705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:37:26.088234 705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/wal
uuid: "92e09dcddb5b4847ba244f935e25a660"
format_stamp: "Formatted at 2026-08-06 18:37:26 on dist-test-slave-8w9v"
server_key: "567a1a5ac4b8ef7402e00d1c7c6acf39"
server_key_iv: "4aa82bdfe74a165d6c7801d19bfacb6c"
server_key_version: "encryptionkey@0"
I20260806 18:37:26.088351 705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:26.107220 705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:26.110874 705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:26.111052 705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:26.116176 622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } attempt: 3
I20260806 18:37:26.117063 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.116743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55874)
0806 18:37:26.116920 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:26.116984 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 18:37:26.117969 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.117731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55880)
0806 18:37:26.117855 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:26.117911 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:37:26.118134 622 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:36307: Network error: Client connection negotiation failed: client connection to 127.18.125.188:36307: connect: Connection refused (error 111)
I20260806 18:37:26.119210 705 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.188:36307
I20260806 18:37:26.119525 775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.188:36307 every 8 connection(s)
I20260806 18:37:26.120333 705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
I20260806 18:37:26.123765 776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:26.127842 776 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:37:26.128774 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 705
I20260806 18:37:26.128911 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041269054381-18934-0/minicluster-data/master-2/wal/instance
I20260806 18:37:26.129202 18934 external_mini_cluster.cc:1428] Setting key 7c503070ee92c55e28ca27365640e513
I20260806 18:37:26.136897 699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } attempt: 3
Aug 06 18:37:26 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
Aug 06 18:37:26 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041446, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
Aug 06 18:37:26 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
I20260806 18:37:26.144891 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.130233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33636 (local address 127.18.125.190:36583)
0806 18:37:26.130395 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:26.130400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:26.131262 (+ 862us) server_negotiation.cc:407] Connection header received
0806 18:37:26.132225 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:26.132229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:26.132297 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:26.132397 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:26.133288 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.134136 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.135580 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.135915 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.136049 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.140563 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:26.140569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:26.140572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:26.140607 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:26.142599 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.143167 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.143171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.143172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.143273 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.143730 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.143734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.143736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.143965 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:26.144072 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:26.144563 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 18:37:26.144702 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:37:26.145812 780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.130626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:36583 (local address 127.0.0.1:33636)
0806 18:37:26.131112 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:26.131156 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:26.131988 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:26.132452 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:26.132458 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:26.132845 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:26.133130 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:26.133143 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.134281 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.134283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:26.135438 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:26.135450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.136250 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.136254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:26.136480 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.138413 (+ 1933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:26.138445 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:26.140368 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:26.142714 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:26.142724 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:26.142737 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:26.143052 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:26.143464 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:26.143467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:26.143469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:26.143626 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:26.144076 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:26.144086 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:26.144299 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 18:37:26.144510 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 18:37:26.144871 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":3,"thread_start_us":117,"threads_started":1}
I20260806 18:37:26.146081 781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.131468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55892 (local address 127.18.125.188:36307)
0806 18:37:26.131744 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:26.131754 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:26.131788 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:37:26.132003 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:26.132025 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:26.132272 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:26.132499 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:26.133389 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.134556 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.135741 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.136391 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.136482 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.140353 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:26.140359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:26.140371 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:26.140395 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:26.142499 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.143066 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.143074 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.143076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.143230 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.143596 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.143600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.143602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.144230 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:26.144473 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:26.144684 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 18:37:26.145096 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":251,"server-negotiator.queue_time_us":181,"thread_start_us":99,"threads_started":1}
I20260806 18:37:26.148039 776 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:37:26.149477 703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.137515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55904)
0806 18:37:26.137736 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:26.137749 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:26.137825 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:26.138305 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:26.138308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:26.138520 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:26.138651 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:26.138661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.139836 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.139841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:26.140698 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:26.140715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.141589 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.141592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:26.141661 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.142053 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:26.142071 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:26.143504 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:26.147641 (+ 4137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:26.147645 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:26.147647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:26.147952 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:26.148344 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:26.148347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:26.148349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:26.148470 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:26.148944 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:26.148948 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:26.149066 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 18:37:26.149195 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 18:37:26.149353 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260806 18:37:26.149575 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.137678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55904 (local address 127.18.125.188:36307)
0806 18:37:26.138088 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:26.138095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:26.138106 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:37:26.138135 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:26.138137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:26.138171 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:26.138278 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:26.138808 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.139677 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.140844 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.141161 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.141289 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.143663 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:26.143667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:26.143672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:26.143702 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:26.147502 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.148076 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.148079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.148081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.148236 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.148603 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.148607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.148609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.148851 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:26.148956 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:26.149255 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 18:37:26.149387 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":198,"threads_started":1}
I20260806 18:37:26.153335 699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58: Bootstrap starting.
Aug 06 18:37:26 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041446, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
Aug 06 18:37:26 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:26.156361 699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:26.157410 699 log.cc:824] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:26.159689 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.149637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33644 (local address 127.18.125.190:36583)
0806 18:37:26.149985 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:26.149989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:26.150004 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:37:26.150124 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:26.150127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:26.150177 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:26.150261 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:26.151033 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.151524 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.152558 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.152850 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.152965 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.155658 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:26.155663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:26.155665 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:26.155694 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:26.157698 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.158250 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.158254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.158256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.158350 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.158706 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.158709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.158712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.158996 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:26.159250 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:26.159396 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 18:37:26.159530 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260}
I20260806 18:37:26.159979 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.148764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39130 (local address 127.18.125.189:38213)
0806 18:37:26.148881 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:26.148885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:26.149024 (+ 139us) server_negotiation.cc:407] Connection header received
0806 18:37:26.149585 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:26.149592 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:26.149652 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:26.149739 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:26.150220 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.150946 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.151992 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.152370 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.152499 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.156114 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:26.156159 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:26.156164 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:26.156199 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:26.158000 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.158522 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.158526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.158528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.158638 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.158979 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.158983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.158984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.159202 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:26.159296 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:26.159673 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 18:37:26.159820 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:37:26.160466 780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.148678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:38213 (local address 127.0.0.1:39130)
0806 18:37:26.148956 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:26.148973 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:26.149056 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:26.149748 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:26.149751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:26.149922 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:26.150091 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:26.150127 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.151090 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.151093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:26.151856 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:26.151868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.152506 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.152510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:26.152616 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.153334 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:26.153353 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:26.155926 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:26.158127 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:26.158130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:26.158132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:26.158418 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:26.158753 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:26.158759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:26.158764 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:26.158863 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:26.159346 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:26.159349 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:26.159467 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 18:37:26.160141 (+ 674us) client_negotiation.cc:240] Negotiation successful
0806 18:37:26.160309 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260806 18:37:26.160722 699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58: No bootstrap required, opened a new log
I20260806 18:37:26.161427 776 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:37:26.165583 699 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } }
I20260806 18:37:26.165879 699 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:26.165942 699 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8265cf13d014e64a59872bd395e3d58, State: Initialized, Role: FOLLOWER
Aug 06 18:37:26 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
I20260806 18:37:26.166455 699 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } }
Aug 06 18:37:26 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041446, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
I20260806 18:37:26.167004 784 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } } }
I20260806 18:37:26.167146 784 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:37:26.168042 699 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:26.171442 780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.161911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55918)
0806 18:37:26.162276 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:26.162288 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:26.162374 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:26.162622 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:26.162625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:26.162808 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:26.162904 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:26.162910 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.163905 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.163908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:26.164700 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:26.164715 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.165216 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.165219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:26.165356 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.165825 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:26.165904 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:26.167434 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:26.169608 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:26.169615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:26.169617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:26.169850 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:26.170184 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:26.170187 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:26.170189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:26.170298 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:26.170887 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:26.170890 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:26.170986 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 18:37:26.171088 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 18:37:26.171238 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":37}
I20260806 18:37:26.172092 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.161200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39146 (local address 127.18.125.189:38213)
0806 18:37:26.161297 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:26.161301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:26.161314 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:37:26.161436 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:26.161440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:26.161483 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:26.161584 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:26.162230 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.162907 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.163661 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.163938 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.164040 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.167598 (+ 3558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:26.167602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:26.167605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:26.167634 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:26.169703 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.170199 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.170203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.170205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.170319 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.170645 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.170648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.170650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.170880 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:26.171063 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:26.171825 (+ 762us) server_negotiation.cc:299] Negotiation successful
0806 18:37:26.171950 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 18:37:26.172545 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.162117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55918 (local address 127.18.125.188:36307)
0806 18:37:26.162213 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:26.162217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:26.162361 (+ 144us) server_negotiation.cc:407] Connection header received
0806 18:37:26.162460 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:26.162462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:26.162508 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:26.162586 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:26.163037 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.163780 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.164854 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.165106 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.165219 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.167590 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:26.167593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:26.167596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:26.167624 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:26.169501 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.169962 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.169965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.169967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.170042 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.170407 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.170412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.170413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.170743 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:26.170850 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:26.172023 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0806 18:37:26.172392 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":31}
W20260806 18:37:26.174480 796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:37:26.174567 796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:37:26.172364 629 master_runner.cc:186] Error getting master registration for 127.18.125.190:36583: OK, instance_id { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" instance_seqno: 1786041445637284 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:37:26.175362 629 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:37:26.178285 776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660: Bootstrap starting.
I20260806 18:37:26.178416 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:37:26.181025 776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:26.182565 776 log.cc:824] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:26.185923 776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660: No bootstrap required, opened a new log
I20260806 18:37:26.194713 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.181010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.18.125.188:36307)
0806 18:37:26.181143 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:26.181148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:26.182315 (+ 1167us) server_negotiation.cc:407] Connection header received
0806 18:37:26.182529 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:26.182532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:26.182591 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:26.182648 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:26.183760 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.184425 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.185574 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.185897 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.185991 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.188363 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:26.188368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:26.188371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:26.188402 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:26.190895 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.191524 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.191528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.191530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.191643 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.192292 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.192295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.192297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.192663 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:26.192808 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:26.194387 (+ 1579us) server_negotiation.cc:299] Negotiation successful
0806 18:37:26.194544 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:37:26.194523 776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } }
I20260806 18:37:26.195128 776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:26.195209 776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e09dcddb5b4847ba244f935e25a660, State: Initialized, Role: FOLLOWER
I20260806 18:37:26.195904 776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } }
I20260806 18:37:26.197944 776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:26.200424 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.181010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39154 (local address 127.18.125.189:38213)
0806 18:37:26.181144 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:26.181149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:26.184462 (+ 3313us) server_negotiation.cc:407] Connection header received
0806 18:37:26.184654 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:26.184658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:26.184722 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:26.184781 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:26.185875 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.186690 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.188429 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.188785 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.188867 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.195835 (+ 6968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:26.195840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:26.195843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:26.195881 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:26.197885 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.198612 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.198616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.198618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.198723 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.199108 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.199114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.199119 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.199439 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:26.199584 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:26.200005 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 18:37:26.200208 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:37:26.204403 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.182049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33656 (local address 127.18.125.190:36583)
0806 18:37:26.182277 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:26.182283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:26.189773 (+ 7490us) server_negotiation.cc:407] Connection header received
0806 18:37:26.190004 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:26.190008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:26.190073 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:26.190128 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:26.191278 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.192418 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.194326 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.194771 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.194878 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.199098 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:26.199103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:26.199107 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:26.199145 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:26.201024 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.202888 (+ 1864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.202892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.202894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.202993 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.203298 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.203302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.203304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.203556 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:26.203663 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:26.204062 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 18:37:26.204223 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
W20260806 18:37:26.205065 817 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:37:26.205134 817 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:37:26.202968 705 master_runner.cc:186] Error getting master registration for 127.18.125.190:36583: OK, instance_id { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" instance_seqno: 1786041445637284 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:37:26.206548 705 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:37:26.208367 806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } } }
I20260806 18:37:26.208513 806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:37:26.282322 622 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } attempt: 4
Aug 06 18:37:26 dist-test-slave-8w9v krb5kdc[539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
I20260806 18:37:26.294001 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.283032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:36307 (local address 127.0.0.1:55928)
0806 18:37:26.283392 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:26.283411 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:26.283511 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:26.284258 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:26.284261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:26.284497 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:26.284617 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:26.284624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.285717 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.285721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:26.286567 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:26.286579 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.287226 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.287230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:26.287329 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.287970 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:26.287988 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:26.289915 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:26.292176 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:26.292179 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:26.292181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:26.292480 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:26.292847 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:26.292850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:26.292852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:26.292956 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:26.293442 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:26.293445 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:26.293551 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 18:37:26.293644 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:37:26.293788 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259}
I20260806 18:37:26.294533 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:26.283679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55928 (local address 127.18.125.188:36307)
0806 18:37:26.283819 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:26.283825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:26.283841 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:37:26.283892 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:26.283896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:26.283945 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:26.284049 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:26.284772 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.285564 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.286719 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:26.287079 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:26.287191 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:26.290092 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:26.290097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:26.290101 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:26.290132 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:26.292021 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.292572 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.292576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.292578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.292679 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:26.293065 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:26.293069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:26.293071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:26.293337 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:26.293448 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:26.293743 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 18:37:26.293874 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:37:26.298280 622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd: Bootstrap starting.
I20260806 18:37:26.301352 622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:26.302301 622 log.cc:824] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:26.304716 622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd: No bootstrap required, opened a new log
I20260806 18:37:26.309027 622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } }
I20260806 18:37:26.309283 622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:26.309346 622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c661ffdb52d40fd87132252fe507edd, State: Initialized, Role: FOLLOWER
I20260806 18:37:26.309860 622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } }
I20260806 18:37:26.310446 820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } } }
I20260806 18:37:26.310668 820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:37:26.311347 622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [sys.catalog]: configured and running, proceeding with master startup.
W20260806 18:37:26.316970 831 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:37:26.317149 831 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:37:26.321013 552 master_runner.cc:263] No leader master found from master 2c661ffdb52d40fd87132252fe507edd
I20260806 18:37:26.321105 552 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 18:37:26.362571 727 tablet.cc:2369] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:37:26.372776 652 tablet.cc:2369] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:37:26.383711 573 tablet.cc:2369] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:37:26.491438 784 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:26.491683 784 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } }
I20260806 18:37:26.492836 784 leader_election.cc:288] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c661ffdb52d40fd87132252fe507edd (127.18.125.190:36583), 92e09dcddb5b4847ba244f935e25a660 (127.18.125.188:36307)
I20260806 18:37:26.493522 750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8265cf13d014e64a59872bd395e3d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e09dcddb5b4847ba244f935e25a660" is_pre_election: true
I20260806 18:37:26.493562 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8265cf13d014e64a59872bd395e3d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c661ffdb52d40fd87132252fe507edd" is_pre_election: true
I20260806 18:37:26.493950 595 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8265cf13d014e64a59872bd395e3d58 in term 0.
I20260806 18:37:26.493911 750 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8265cf13d014e64a59872bd395e3d58 in term 0.
I20260806 18:37:26.494505 648 leader_election.cc:302] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c661ffdb52d40fd87132252fe507edd, b8265cf13d014e64a59872bd395e3d58; no voters:
I20260806 18:37:26.494917 784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:37:26.495149 784 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:37:26.495254 784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:26.496698 784 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } }
I20260806 18:37:26.497139 784 leader_election.cc:288] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [CANDIDATE]: Term 1 election: Requested vote from peers 2c661ffdb52d40fd87132252fe507edd (127.18.125.190:36583), 92e09dcddb5b4847ba244f935e25a660 (127.18.125.188:36307)
I20260806 18:37:26.497579 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8265cf13d014e64a59872bd395e3d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c661ffdb52d40fd87132252fe507edd"
I20260806 18:37:26.497584 750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8265cf13d014e64a59872bd395e3d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e09dcddb5b4847ba244f935e25a660"
I20260806 18:37:26.497735 750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:26.497733 595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:26.499070 750 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8265cf13d014e64a59872bd395e3d58 in term 1.
I20260806 18:37:26.499070 595 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8265cf13d014e64a59872bd395e3d58 in term 1.
I20260806 18:37:26.499445 648 leader_election.cc:302] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c661ffdb52d40fd87132252fe507edd, b8265cf13d014e64a59872bd395e3d58; no voters:
I20260806 18:37:26.499657 784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:26.499902 784 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [term 1 LEADER]: Becoming Leader. State: Replica: b8265cf13d014e64a59872bd395e3d58, State: Running, Role: LEADER
I20260806 18:37:26.500301 784 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } }
I20260806 18:37:26.502475 833 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8265cf13d014e64a59872bd395e3d58. Latest consensus state: current_term: 1 leader_uuid: "b8265cf13d014e64a59872bd395e3d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } } }
I20260806 18:37:26.502624 833 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:26.503041 835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:37:26.503894 835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:37:26.509249 595 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [term 1 FOLLOWER]: Refusing update from remote peer b8265cf13d014e64a59872bd395e3d58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:26.509249 750 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [term 1 FOLLOWER]: Refusing update from remote peer b8265cf13d014e64a59872bd395e3d58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:26.510035 833 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:26.510226 784 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:26.515241 806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8265cf13d014e64a59872bd395e3d58. Latest consensus state: current_term: 1 leader_uuid: "b8265cf13d014e64a59872bd395e3d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } } }
I20260806 18:37:26.515232 820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8265cf13d014e64a59872bd395e3d58. Latest consensus state: current_term: 1 leader_uuid: "b8265cf13d014e64a59872bd395e3d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } } }
I20260806 18:37:26.515419 820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:37:26.515420 806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:37:26.518487 833 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8265cf13d014e64a59872bd395e3d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } } }
I20260806 18:37:26.518647 833 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:26.518602 784 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8265cf13d014e64a59872bd395e3d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } } }
I20260806 18:37:26.518731 784 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:26.518883 838 mvcc.cc:205] Tried to move back new op lower bound from 7315625764895047680 to 7315625764868345856. Current Snapshot: MvccSnapshot[applied={T|T < 7315625764895047680}]
I20260806 18:37:26.519033 820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8265cf13d014e64a59872bd395e3d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } } }
I20260806 18:37:26.519174 820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:37:26.519320 836 mvcc.cc:205] Tried to move back new op lower bound from 7315625764895047680 to 7315625764868345856. Current Snapshot: MvccSnapshot[applied={T|T < 7315625764895047680}]
I20260806 18:37:26.519883 806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8265cf13d014e64a59872bd395e3d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c661ffdb52d40fd87132252fe507edd" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36583 } } peers { permanent_uuid: "b8265cf13d014e64a59872bd395e3d58" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 38213 } } peers { permanent_uuid: "92e09dcddb5b4847ba244f935e25a660" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 36307 } } }
I20260806 18:37:26.520016 806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:37:26.520475 837 mvcc.cc:205] Tried to move back new op lower bound from 7315625764895047680 to 7315625764868345856. Current Snapshot: MvccSnapshot[applied={T|T < 7315625764895047680}]
I20260806 18:37:26.522471 835 catalog_manager.cc:1347] Generated new cluster ID: 8268ce141ed2468394ce646c540f8947
I20260806 18:37:26.522536 835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:37:26.549008 835 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:37:26.550386 835 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:37:26.565097 835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8265cf13d014e64a59872bd395e3d58: Generated new TSK 0
I20260806 18:37:26.566236 835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:37:27.206555 817 catalog_manager.cc:1259] Loaded cluster ID: 8268ce141ed2468394ce646c540f8947
I20260806 18:37:27.206692 817 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660: loading cluster ID for follower catalog manager: success
I20260806 18:37:27.208448 817 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660: acquiring CA information for follower catalog manager: success
I20260806 18:37:27.209671 817 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 92e09dcddb5b4847ba244f935e25a660: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:37:27.318626 831 catalog_manager.cc:1259] Loaded cluster ID: 8268ce141ed2468394ce646c540f8947
I20260806 18:37:27.318717 831 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd: loading cluster ID for follower catalog manager: success
I20260806 18:37:27.320469 831 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd: acquiring CA information for follower catalog manager: success
I20260806 18:37:27.321738 831 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2c661ffdb52d40fd87132252fe507edd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:37:27.645958 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 552
I20260806 18:37:27.653498 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 629
I20260806 18:37:27.661098 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 705
2026-08-06T18:37:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3448 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:37:27 dist-test-slave-8w9v krb5kdc[850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:37:27 dist-test-slave-8w9v krb5kdc[850](info): set up 2 sockets
Aug 06 18:37:27 dist-test-slave-8w9v krb5kdc[850](info): commencing operation
krb5kdc: starting...
W20260806 18:37:28.746917 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:37:28 dist-test-slave-8w9v krb5kdc[850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:37:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:37:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:37:28.884369 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:32939
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:32939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:29.006666 863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:29.006984 863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:29.007027 863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:29.011296 863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:29.011374 863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:29.011394 863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:29.011418 863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:29.011435 863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:29.017148 863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:32939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:32939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:29.018407 863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:29.019549 863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:29.029076 869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:29.029058 868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:29.029328 863 server_base.cc:1034] running on GCE node
W20260806 18:37:29.029096 871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:29.030084 863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:29.031481 863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:29.032737 863 hybrid_clock.cc:648] HybridClock initialized: now 1786041449032711 us; error 41 us; skew 500 ppm
Aug 06 18:37:29 dist-test-slave-8w9v krb5kdc[850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:29.036218 863 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:37:29.037331 863 webserver.cc:460] Webserver started at http://127.18.125.190:35525/ using document root <none> and password file <none>
I20260806 18:37:29.038007 863 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:29.038082 863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:29.038341 863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:29.040877 863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "4b9b4210e2a142e88bc3842c107c791d"
format_stamp: "Formatted at 2026-08-06 18:37:29 on dist-test-slave-8w9v"
server_key: "7f57834d5cfcac971d73cfe025ec02e2"
server_key_iv: "773e28ac1306fe9584f5bcce6dc317e0"
server_key_version: "encryptionkey@0"
I20260806 18:37:29.041604 863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "4b9b4210e2a142e88bc3842c107c791d"
format_stamp: "Formatted at 2026-08-06 18:37:29 on dist-test-slave-8w9v"
server_key: "7f57834d5cfcac971d73cfe025ec02e2"
server_key_iv: "773e28ac1306fe9584f5bcce6dc317e0"
server_key_version: "encryptionkey@0"
I20260806 18:37:29.046518 863 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 18:37:29.049399 878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:29.050988 863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:29.051185 863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "4b9b4210e2a142e88bc3842c107c791d"
format_stamp: "Formatted at 2026-08-06 18:37:29 on dist-test-slave-8w9v"
server_key: "7f57834d5cfcac971d73cfe025ec02e2"
server_key_iv: "773e28ac1306fe9584f5bcce6dc317e0"
server_key_version: "encryptionkey@0"
I20260806 18:37:29.051331 863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:29.071341 863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:29.075080 863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:29.075305 863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:29.083052 863 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:32939
I20260806 18:37:29.083072 932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:32939 every 8 connection(s)
I20260806 18:37:29.084115 863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:37:29.087201 933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:29.092800 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 863
I20260806 18:37:29.092964 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:37:29.093263 18934 external_mini_cluster.cc:1428] Setting key 557da96776d686bd3759e5ca0fc628c8
I20260806 18:37:29.093819 933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d: Bootstrap starting.
I20260806 18:37:29.096657 933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:29.097484 933 log.cc:824] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:29.099869 933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d: No bootstrap required, opened a new log
Aug 06 18:37:29 dist-test-slave-8w9v krb5kdc[850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:29.104552 933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9b4210e2a142e88bc3842c107c791d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32939 } }
I20260806 18:37:29.104910 933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:29.104990 933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9b4210e2a142e88bc3842c107c791d, State: Initialized, Role: FOLLOWER
I20260806 18:37:29.105608 933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9b4210e2a142e88bc3842c107c791d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32939 } }
I20260806 18:37:29.105795 933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:37:29.105865 933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:37:29.105968 933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:29.107255 933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9b4210e2a142e88bc3842c107c791d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32939 } }
I20260806 18:37:29.107647 933 leader_election.cc:302] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b9b4210e2a142e88bc3842c107c791d; no voters:
I20260806 18:37:29.108033 933 leader_election.cc:288] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:37:29.108188 938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:29.108474 938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [term 1 LEADER]: Becoming Leader. State: Replica: 4b9b4210e2a142e88bc3842c107c791d, State: Running, Role: LEADER
I20260806 18:37:29.108903 938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9b4210e2a142e88bc3842c107c791d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32939 } }
I20260806 18:37:29.109508 933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:29.109714 936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:29.094813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50196 (local address 127.18.125.190:32939)
0806 18:37:29.095330 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:29.095346 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:29.095385 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:37:29.096089 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:29.096106 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:29.096523 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:29.096873 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:29.097756 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:29.099182 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:29.100231 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:29.100918 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:29.101110 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:29.104014 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:29.104022 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:29.104043 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:29.104080 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:29.106342 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:29.106807 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:29.106813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:29.106820 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:29.106993 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:29.107279 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:29.107283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:29.107285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:29.107843 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:29.108069 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:29.108416 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 18:37:29.108727 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":161,"threads_started":1}
I20260806 18:37:29.110208 939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b9b4210e2a142e88bc3842c107c791d. Latest consensus state: current_term: 1 leader_uuid: "4b9b4210e2a142e88bc3842c107c791d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9b4210e2a142e88bc3842c107c791d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32939 } } }
I20260806 18:37:29.110369 939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:29.110803 945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:37:29.111518 940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b9b4210e2a142e88bc3842c107c791d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9b4210e2a142e88bc3842c107c791d" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 32939 } } }
I20260806 18:37:29.112372 940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:29.113056 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:37:29.115540 945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:37:29.121482 945 catalog_manager.cc:1347] Generated new cluster ID: 62212881d73746f6ae1c9ee2309431da
I20260806 18:37:29.121582 945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:37:29.130606 945 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:37:29.131578 945 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:37:29.147684 945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b9b4210e2a142e88bc3842c107c791d: Generated new TSK 0
I20260806 18:37:29.148586 945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:37:29.169452 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 863
2026-08-06T18:37:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1505 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:37:29 dist-test-slave-8w9v krb5kdc[961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:37:29 dist-test-slave-8w9v krb5kdc[961](info): set up 2 sockets
Aug 06 18:37:29 dist-test-slave-8w9v krb5kdc[961](info): commencing operation
krb5kdc: starting...
W20260806 18:37:30.245677 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.006s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:37:30 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:37:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:37:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:37:30.385836 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34987
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:34987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:30.505889 974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:30.506234 974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:30.506317 974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:30.510409 974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:30.510520 974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:30.510561 974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:30.510619 974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:30.510660 974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:30.516096 974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:34987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:30.517297 974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:30.518378 974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:30.527704 980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:30.527923 982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:30.527704 979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:30.529016 974 server_base.cc:1034] running on GCE node
I20260806 18:37:30.529727 974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:30.530982 974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:30.532220 974 hybrid_clock.cc:648] HybridClock initialized: now 1786041450532203 us; error 38 us; skew 500 ppm
Aug 06 18:37:30 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:30.538569 974 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:37:30.539592 974 webserver.cc:460] Webserver started at http://127.18.125.190:35535/ using document root <none> and password file <none>
I20260806 18:37:30.540230 974 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:30.540303 974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:30.540496 974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:30.542655 974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5"
format_stamp: "Formatted at 2026-08-06 18:37:30 on dist-test-slave-8w9v"
server_key: "5507aadaec10b2cd51a5f9cba60470ed"
server_key_iv: "f92faa77fb1da25754e8a8a2e8962c56"
server_key_version: "encryptionkey@0"
I20260806 18:37:30.543195 974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5"
format_stamp: "Formatted at 2026-08-06 18:37:30 on dist-test-slave-8w9v"
server_key: "5507aadaec10b2cd51a5f9cba60470ed"
server_key_iv: "f92faa77fb1da25754e8a8a2e8962c56"
server_key_version: "encryptionkey@0"
I20260806 18:37:30.547263 974 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:37:30.550145 989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:30.551520 974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 18:37:30.551671 974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5"
format_stamp: "Formatted at 2026-08-06 18:37:30 on dist-test-slave-8w9v"
server_key: "5507aadaec10b2cd51a5f9cba60470ed"
server_key_iv: "f92faa77fb1da25754e8a8a2e8962c56"
server_key_version: "encryptionkey@0"
I20260806 18:37:30.551798 974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:30.573999 974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:30.577780 974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:30.578027 974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:30.586380 974 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:34987
I20260806 18:37:30.586727 1042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:34987 every 8 connection(s)
I20260806 18:37:30.587664 974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:37:30.591018 1043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:30.593942 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 974
I20260806 18:37:30.594092 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:37:30.594403 18934 external_mini_cluster.cc:1428] Setting key 7f2d80f0c63a98e77b8fd3e18c2e5ac7
I20260806 18:37:30.598260 1043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Bootstrap starting.
I20260806 18:37:30.600865 1043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:30.601686 1043 log.cc:824] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:37:30 dist-test-slave-8w9v krb5kdc[961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:30.604733 1043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: No bootstrap required, opened a new log
I20260806 18:37:30.608650 1043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:30.608951 1043 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:30.609047 1043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56d6d2a8b96b45a1a1068d39f55f1dc5, State: Initialized, Role: FOLLOWER
I20260806 18:37:30.609579 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:30.596161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40998 (local address 127.18.125.190:34987)
0806 18:37:30.596660 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:30.596676 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:30.596715 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:37:30.597399 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:30.597417 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:30.597789 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:30.598168 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:30.599072 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:30.600193 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:30.601141 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:30.601638 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:30.601835 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:30.604528 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:30.604533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:30.604549 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:30.604581 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:30.606436 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:30.606934 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:30.606940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:30.606945 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:30.607066 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:30.607369 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:30.607373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:30.607375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:30.607809 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:30.608039 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:30.608362 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 18:37:30.608676 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":140,"threads_started":1}
I20260806 18:37:30.609561 1043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:30.609725 1043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:37:30.609794 1043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:37:30.609920 1043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:30.611179 1043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:30.611547 1043 leader_election.cc:302] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56d6d2a8b96b45a1a1068d39f55f1dc5; no voters:
I20260806 18:37:30.611905 1043 leader_election.cc:288] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:37:30.611985 1048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:30.612190 1048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 1 LEADER]: Becoming Leader. State: Replica: 56d6d2a8b96b45a1a1068d39f55f1dc5, State: Running, Role: LEADER
I20260806 18:37:30.612490 1048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:30.613303 1043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:30.614363 1049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56d6d2a8b96b45a1a1068d39f55f1dc5. Latest consensus state: current_term: 1 leader_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } } }
I20260806 18:37:30.614401 1050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } } }
I20260806 18:37:30.614529 1049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:30.614529 1050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:30.615188 1058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:37:30.618824 1058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:37:30.625599 1058 catalog_manager.cc:1347] Generated new cluster ID: fab6bf16aae84f929efc66dad8eff566
I20260806 18:37:30.625718 1058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:37:30.639429 1058 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:37:30.640878 1058 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:37:30.647468 1058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Generated new TSK 0
I20260806 18:37:30.648314 1058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:37:30.688629 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34987
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:30.811628 1070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:30.812450 1070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:30.812604 1070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:30.819861 1070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:30.819979 1070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:30.820058 1070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:37:30.826216 1070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:30.827351 1070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:30.828402 1070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:37:30.841542 1070 server_base.cc:1034] running on GCE node
W20260806 18:37:30.841509 1078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:30.841471 1076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:30.841475 1075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:30.842310 1070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:30.843026 1070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:30.844204 1070 hybrid_clock.cc:648] HybridClock initialized: now 1786041450844165 us; error 52 us; skew 500 ppm
Aug 06 18:37:30 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:30.847074 1070 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:37:30.848176 1070 webserver.cc:460] Webserver started at http://127.18.125.129:38809/ using document root <none> and password file <none>
I20260806 18:37:30.848829 1070 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:30.848899 1070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:30.849164 1070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:30.851459 1070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "43982f4af9314e91b1e68e0f20ad74c9"
format_stamp: "Formatted at 2026-08-06 18:37:30 on dist-test-slave-8w9v"
server_key: "ba7bba57a2da97ec48ece151de8244a0"
server_key_iv: "6e872a69fe6356a35aa18173e6664e39"
server_key_version: "encryptionkey@0"
I20260806 18:37:30.852082 1070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "43982f4af9314e91b1e68e0f20ad74c9"
format_stamp: "Formatted at 2026-08-06 18:37:30 on dist-test-slave-8w9v"
server_key: "ba7bba57a2da97ec48ece151de8244a0"
server_key_iv: "6e872a69fe6356a35aa18173e6664e39"
server_key_version: "encryptionkey@0"
I20260806 18:37:30.858117 1070 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 18:37:30.861186 1085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:30.862722 1070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:30.863175 1070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "43982f4af9314e91b1e68e0f20ad74c9"
format_stamp: "Formatted at 2026-08-06 18:37:30 on dist-test-slave-8w9v"
server_key: "ba7bba57a2da97ec48ece151de8244a0"
server_key_iv: "6e872a69fe6356a35aa18173e6664e39"
server_key_version: "encryptionkey@0"
I20260806 18:37:30.863374 1070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:30.880177 1070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:30.883932 1070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:30.884246 1070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:30.884958 1070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:30.885886 1070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:30.885959 1070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:30.886040 1070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:30.886091 1070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:30.895776 1070 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:43067
I20260806 18:37:30.895799 1199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:43067 every 8 connection(s)
I20260806 18:37:30.896859 1070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:37:30.897209 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 1070
I20260806 18:37:30.897387 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:37:30.897735 18934 external_mini_cluster.cc:1428] Setting key 9051907d88f0bdc662c6cb7bf4a86e8a
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
Aug 06 18:37:30 dist-test-slave-8w9v krb5kdc[961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
I20260806 18:37:30.913440 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:30.899284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:48193 (local address 127.18.125.190:34987)
0806 18:37:30.899453 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:30.899459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:30.900648 (+ 1189us) server_negotiation.cc:407] Connection header received
0806 18:37:30.902218 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:30.902222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:30.902292 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:30.902419 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:30.903842 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:30.904522 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:30.905906 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:30.906168 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:30.906317 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:30.909496 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:30.909499 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:30.909502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:30.909529 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:30.910899 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:30.911534 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:30.911537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:30.911538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:30.911622 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:30.912072 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:30.912075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:30.912076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:30.912361 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:30.912482 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:30.913100 (+ 618us) server_negotiation.cc:299] Negotiation successful
0806 18:37:30.913258 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 18:37:30.914804 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:30.899790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:48193)
0806 18:37:30.900455 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:30.900526 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:30.901954 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:30.902627 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:30.902636 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:30.903076 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:30.903642 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:30.903659 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:30.904698 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:30.904707 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:30.905738 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:30.905751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:30.906376 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:30.906381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:30.906584 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:30.907413 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:30.907433 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:30.909290 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:30.911060 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:30.911066 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:30.911081 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:30.911429 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:30.911718 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:30.911720 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:30.911722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:30.911971 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:30.912484 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:30.912494 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:30.912790 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 18:37:30.913055 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 18:37:30.913434 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":202,"threads_started":1}
I20260806 18:37:30.916653 1200 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34987
I20260806 18:37:30.916977 1200 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:30.917575 1200 heartbeater.cc:511] Master 127.18.125.190:34987 requested a full tablet report, sending...
I20260806 18:37:30.919456 1006 ts_manager.cc:194] Registered new tserver with Master: 43982f4af9314e91b1e68e0f20ad74c9 (127.18.125.129:43067)
I20260806 18:37:30.920780 1006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:48193
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:37:30.931388 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34987
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:31.055974 1206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:31.056384 1206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:31.056445 1206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:31.060683 1206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:31.060751 1206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:31.060813 1206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:37:31.066212 1206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:31.067382 1206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:31.068616 1206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:31.078773 1212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.078776 1211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.078779 1214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:31.079053 1206 server_base.cc:1034] running on GCE node
I20260806 18:37:31.079701 1206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:31.080461 1206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:31.081627 1206 hybrid_clock.cc:648] HybridClock initialized: now 1786041451081591 us; error 48 us; skew 500 ppm
Aug 06 18:37:31 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:31.084745 1206 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:37:31.085686 1206 webserver.cc:460] Webserver started at http://127.18.125.130:34285/ using document root <none> and password file <none>
I20260806 18:37:31.086282 1206 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:31.086345 1206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:31.086566 1206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:31.088773 1206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "5498db8406b44b3aa0657eeb79e81479"
format_stamp: "Formatted at 2026-08-06 18:37:31 on dist-test-slave-8w9v"
server_key: "91cb82edd7cee5b38247dfb6aba9f37b"
server_key_iv: "a0bcdc75999954ae770a70f8c029b2ed"
server_key_version: "encryptionkey@0"
I20260806 18:37:31.089381 1206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "5498db8406b44b3aa0657eeb79e81479"
format_stamp: "Formatted at 2026-08-06 18:37:31 on dist-test-slave-8w9v"
server_key: "91cb82edd7cee5b38247dfb6aba9f37b"
server_key_iv: "a0bcdc75999954ae770a70f8c029b2ed"
server_key_version: "encryptionkey@0"
I20260806 18:37:31.093452 1206 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:37:31.096238 1221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:31.097657 1206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:31.097832 1206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "5498db8406b44b3aa0657eeb79e81479"
format_stamp: "Formatted at 2026-08-06 18:37:31 on dist-test-slave-8w9v"
server_key: "91cb82edd7cee5b38247dfb6aba9f37b"
server_key_iv: "a0bcdc75999954ae770a70f8c029b2ed"
server_key_version: "encryptionkey@0"
I20260806 18:37:31.097959 1206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:31.113550 1206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:31.117307 1206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:31.117540 1206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:31.118180 1206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:31.119066 1206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:31.119115 1206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:31.119184 1206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:31.119221 1206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:31.128324 1206 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:41465
I20260806 18:37:31.128412 1339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:41465 every 8 connection(s)
I20260806 18:37:31.129303 1206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:37:31.139417 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 1206
I20260806 18:37:31.139521 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:37:31.139814 18934 external_mini_cluster.cc:1428] Setting key bbe1a8c7fde4cf99a86df59c8183d951
Aug 06 18:37:31 dist-test-slave-8w9v krb5kdc[961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:31.145131 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.131258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:56321 (local address 127.18.125.190:34987)
0806 18:37:31.131466 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:31.131471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:31.132384 (+ 913us) server_negotiation.cc:407] Connection header received
0806 18:37:31.133807 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:31.133810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:31.133859 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:31.133937 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:31.135179 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.135730 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.137542 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.137770 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.137860 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.140752 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:31.140757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:31.140760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:31.140794 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:31.142346 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:31.143100 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:31.143103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:31.143104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:31.143182 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:31.143685 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:31.143694 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:31.143699 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:31.143984 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:31.144099 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:31.144749 (+ 650us) server_negotiation.cc:299] Negotiation successful
0806 18:37:31.144910 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 18:37:31.146129 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.131661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:56321)
0806 18:37:31.132215 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:31.132273 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:31.133580 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:31.134106 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:31.134113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:31.134494 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:31.135006 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.135020 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.135865 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.135868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.137404 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.137416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.137933 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.137936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.138103 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.138759 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:31.138776 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:31.140595 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:31.142488 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:31.142498 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:31.142519 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:31.142963 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:31.143282 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:31.143289 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:31.143293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:31.143515 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:31.144111 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:31.144117 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:31.144464 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 18:37:31.144644 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 18:37:31.144959 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":124,"threads_started":1}
I20260806 18:37:31.147513 1340 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34987
I20260806 18:37:31.147832 1340 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:31.148451 1340 heartbeater.cc:511] Master 127.18.125.190:34987 requested a full tablet report, sending...
I20260806 18:37:31.149659 1006 ts_manager.cc:194] Registered new tserver with Master: 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465)
I20260806 18:37:31.150372 1006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:56321
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:37:31.171247 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34987
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:31.289765 1345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:31.290102 1345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:31.290198 1345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:31.294873 1345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:31.294988 1345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:31.295089 1345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:37:31.301007 1345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:31.302296 1345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:31.303445 1345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:31.314040 1350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.314066 1351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.314172 1354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:31.314260 1345 server_base.cc:1034] running on GCE node
I20260806 18:37:31.314958 1345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:31.315786 1345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:31.316970 1345 hybrid_clock.cc:648] HybridClock initialized: now 1786041451316928 us; error 58 us; skew 500 ppm
Aug 06 18:37:31 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:31.320055 1345 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:37:31.321156 1345 webserver.cc:460] Webserver started at http://127.18.125.131:46177/ using document root <none> and password file <none>
I20260806 18:37:31.321753 1345 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:31.321825 1345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:31.322011 1345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:31.324704 1345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "f8920a68a635465ba34d05579edde726"
format_stamp: "Formatted at 2026-08-06 18:37:31 on dist-test-slave-8w9v"
server_key: "fc6d911e81fc6630059a3a62f8a103ff"
server_key_iv: "2a85b5dba8dcd84d9953aa5a8eb22d4f"
server_key_version: "encryptionkey@0"
I20260806 18:37:31.325297 1345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "f8920a68a635465ba34d05579edde726"
format_stamp: "Formatted at 2026-08-06 18:37:31 on dist-test-slave-8w9v"
server_key: "fc6d911e81fc6630059a3a62f8a103ff"
server_key_iv: "2a85b5dba8dcd84d9953aa5a8eb22d4f"
server_key_version: "encryptionkey@0"
I20260806 18:37:31.329260 1345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:37:31.331951 1361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:31.333289 1345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:37:31.333456 1345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "f8920a68a635465ba34d05579edde726"
format_stamp: "Formatted at 2026-08-06 18:37:31 on dist-test-slave-8w9v"
server_key: "fc6d911e81fc6630059a3a62f8a103ff"
server_key_iv: "2a85b5dba8dcd84d9953aa5a8eb22d4f"
server_key_version: "encryptionkey@0"
I20260806 18:37:31.333590 1345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:31.365119 1345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:31.368786 1345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:31.369024 1345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:31.369663 1345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:31.370539 1345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:31.370586 1345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:31.370630 1345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:31.370645 1345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:31.379961 1345 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:36997
I20260806 18:37:31.379998 1476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:36997 every 8 connection(s)
I20260806 18:37:31.380999 1345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:37:31.389747 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 1345
I20260806 18:37:31.389890 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:37:31.390179 18934 external_mini_cluster.cc:1428] Setting key d647bb34abd64c1a2fb01048d28b29d5
Aug 06 18:37:31 dist-test-slave-8w9v krb5kdc[961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:31.395509 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.382987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:55389 (local address 127.18.125.190:34987)
0806 18:37:31.383152 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:31.383157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:31.384039 (+ 882us) server_negotiation.cc:407] Connection header received
0806 18:37:31.385471 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:31.385474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:31.385524 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:31.385612 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:31.386902 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.387478 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.388696 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.388921 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.389010 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.391854 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:31.391857 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:31.391860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:31.391886 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:31.393413 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:31.393909 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:31.393911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:31.393913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:31.393985 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:31.394427 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:31.394430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:31.394431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:31.394615 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:31.394693 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:31.395246 (+ 553us) server_negotiation.cc:299] Negotiation successful
0806 18:37:31.395384 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:37:31.396481 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.383395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:55389)
0806 18:37:31.383873 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:31.383922 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:31.385254 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:31.385777 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:31.385784 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:31.386189 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:31.386720 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.386734 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.387617 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.387620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.388578 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.388588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.389068 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.389071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.389222 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.390056 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:31.390074 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:31.391700 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:31.393528 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:31.393534 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:31.393547 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:31.393821 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:31.394104 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:31.394107 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:31.394109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:31.394321 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:31.394715 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:31.394721 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:31.394986 (+ 265us) client_negotiation.cc:769] Sending connection context
0806 18:37:31.395167 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 18:37:31.395480 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":122,"threads_started":1}
I20260806 18:37:31.397711 1477 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34987
I20260806 18:37:31.398010 1477 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:31.398617 1477 heartbeater.cc:511] Master 127.18.125.190:34987 requested a full tablet report, sending...
I20260806 18:37:31.399925 1006 ts_manager.cc:194] Registered new tserver with Master: f8920a68a635465ba34d05579edde726 (127.18.125.131:36997)
I20260806 18:37:31.400672 1006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:55389
I20260806 18:37:31.405387 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:37:31.417128 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.407552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41002 (local address 127.18.125.190:34987)
0806 18:37:31.407707 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:31.407711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:31.407934 (+ 223us) server_negotiation.cc:407] Connection header received
0806 18:37:31.408267 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:31.408271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:31.408319 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:31.408421 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:31.409195 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.409727 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.410549 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.410763 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.410846 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.412899 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:31.412903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:31.412905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:31.412932 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:31.414856 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:31.415528 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:31.415531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:31.415532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:31.415622 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:31.415972 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:31.415976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:31.415978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:31.416318 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:31.416482 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:31.416761 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 18:37:31.416911 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:37:31.421033 1006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:37:31.424372 1006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:37:31.439693 1488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.433218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:36997 (local address 127.0.0.1:50832)
0806 18:37:31.433696 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:31.433757 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:31.433893 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:31.434759 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:31.434765 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:31.434789 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:31.434940 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.434950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.436364 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.436369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.437955 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.437967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.439114 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.439120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.439230 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.439268 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:37:31.439392 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 18:37:31.439528 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":120,"thread_start_us":91,"threads_started":1}
I20260806 18:37:31.439720 1491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.433916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:41465 (local address 127.0.0.1:36770)
0806 18:37:31.434772 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:31.434787 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:31.434873 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:31.435615 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:31.435619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:31.435638 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:31.435719 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.435724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.437007 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.437013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.438237 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.438247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.439371 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.439376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.439469 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.439488 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:37:31.439563 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:37:31.439621 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260806 18:37:31.439805 1490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.433392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.18.125.131:36997)
0806 18:37:31.434238 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:31.434261 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:31.434286 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:37:31.434370 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:31.434376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:31.434577 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:31.435102 (+ 525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:31.435269 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.436230 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.438110 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.438973 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.439106 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.439248 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:31.439466 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 18:37:31.439615 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":155,"threads_started":1}
I20260806 18:37:31.440353 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.433385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:43067 (local address 127.0.0.1:54520)
0806 18:37:31.433875 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:31.433888 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:31.433962 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:31.434892 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:31.434895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:31.434910 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:31.435048 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.435055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.436557 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.436561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.437955 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.437967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.439894 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.439907 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.440002 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.440038 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 18:37:31.440161 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 18:37:31.440231 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":103,"thread_start_us":104,"threads_started":1}
I20260806 18:37:31.440693 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.434333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36770 (local address 127.18.125.130:41465)
0806 18:37:31.435144 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:31.435164 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:31.435185 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:37:31.435260 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:31.435265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:31.435453 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:31.435600 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:31.435849 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.436879 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.438363 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.439028 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.439175 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.439305 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:31.439640 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 18:37:31.439743 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":178,"threads_started":1}
I20260806 18:37:31.440886 1492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.433996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54520 (local address 127.18.125.129:43067)
0806 18:37:31.434314 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:31.434321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:31.434339 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:37:31.434404 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:31.434408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:31.434577 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:31.434730 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:31.435178 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.436424 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.438114 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.438973 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.439106 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.439261 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:31.440200 (+ 939us) server_negotiation.cc:299] Negotiation successful
0806 18:37:31.440308 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":103,"threads_started":1}
I20260806 18:37:31.443208 1133 tablet_service.cc:1467] Processing CreateTablet for tablet 3541eaa45e8940b4847c4c49e517c2d5 (DEFAULT_TABLE table=test-table [id=e9e0ff4dca5842da96f857dce1f27f1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:31.443312 1271 tablet_service.cc:1467] Processing CreateTablet for tablet 3541eaa45e8940b4847c4c49e517c2d5 (DEFAULT_TABLE table=test-table [id=e9e0ff4dca5842da96f857dce1f27f1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:31.443307 1410 tablet_service.cc:1467] Processing CreateTablet for tablet 3541eaa45e8940b4847c4c49e517c2d5 (DEFAULT_TABLE table=test-table [id=e9e0ff4dca5842da96f857dce1f27f1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:31.444336 1133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3541eaa45e8940b4847c4c49e517c2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:31.444411 1410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3541eaa45e8940b4847c4c49e517c2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:31.444411 1271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3541eaa45e8940b4847c4c49e517c2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:31.451152 1494 tablet_bootstrap.cc:492] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Bootstrap starting.
I20260806 18:37:31.452744 1496 tablet_bootstrap.cc:492] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: Bootstrap starting.
I20260806 18:37:31.452744 1495 tablet_bootstrap.cc:492] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726: Bootstrap starting.
I20260806 18:37:31.453783 1494 tablet_bootstrap.cc:654] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:31.454566 1494 log.cc:824] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:31.455736 1496 tablet_bootstrap.cc:654] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:31.455734 1495 tablet_bootstrap.cc:654] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:31.456729 1496 log.cc:824] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:31.456729 1495 log.cc:824] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:31.457229 1494 tablet_bootstrap.cc:492] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: No bootstrap required, opened a new log
I20260806 18:37:31.457472 1494 ts_tablet_manager.cc:1397] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:37:31.459017 1495 tablet_bootstrap.cc:492] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726: No bootstrap required, opened a new log
I20260806 18:37:31.459216 1496 tablet_bootstrap.cc:492] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: No bootstrap required, opened a new log
I20260806 18:37:31.459235 1495 ts_tablet_manager.cc:1397] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 18:37:31.459412 1496 ts_tablet_manager.cc:1397] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:37:31.461511 1494 raft_consensus.cc:348] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } }
I20260806 18:37:31.461871 1494 raft_consensus.cc:374] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:31.461959 1494 raft_consensus.cc:729] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43982f4af9314e91b1e68e0f20ad74c9, State: Initialized, Role: FOLLOWER
I20260806 18:37:31.462492 1494 consensus_queue.cc:260] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } }
I20260806 18:37:31.463402 1495 raft_consensus.cc:348] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } }
I20260806 18:37:31.463413 1496 raft_consensus.cc:348] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } }
I20260806 18:37:31.463677 1495 raft_consensus.cc:374] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:31.463678 1496 raft_consensus.cc:374] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:31.463711 1494 ts_tablet_manager.cc:1428] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:37:31.463729 1496 raft_consensus.cc:729] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498db8406b44b3aa0657eeb79e81479, State: Initialized, Role: FOLLOWER
I20260806 18:37:31.463740 1495 raft_consensus.cc:729] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8920a68a635465ba34d05579edde726, State: Initialized, Role: FOLLOWER
I20260806 18:37:31.464179 1200 heartbeater.cc:503] Master 127.18.125.190:34987 was elected leader, sending a full tablet report...
I20260806 18:37:31.464265 1495 consensus_queue.cc:260] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } }
I20260806 18:37:31.464745 1496 consensus_queue.cc:260] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } }
I20260806 18:37:31.465019 1477 heartbeater.cc:503] Master 127.18.125.190:34987 was elected leader, sending a full tablet report...
I20260806 18:37:31.465400 1495 ts_tablet_manager.cc:1428] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:37:31.465795 1496 ts_tablet_manager.cc:1428] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:37:31.466120 1340 heartbeater.cc:503] Master 127.18.125.190:34987 was elected leader, sending a full tablet report...
I20260806 18:37:31.494580 1502 raft_consensus.cc:482] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:31.494876 1502 raft_consensus.cc:504] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } }
I20260806 18:37:31.496273 1502 leader_election.cc:288] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8920a68a635465ba34d05579edde726 (127.18.125.131:36997), 43982f4af9314e91b1e68e0f20ad74c9 (127.18.125.129:43067)
I20260806 18:37:31.500538 1490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.496649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:57213 (local address 127.18.125.131:36997)
0806 18:37:31.496813 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:31.496819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:31.496913 (+ 94us) server_negotiation.cc:407] Connection header received
0806 18:37:31.497099 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:31.497104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:31.497170 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:31.497265 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:31.497667 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.498224 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.499390 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.499848 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.499945 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.499988 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:31.500304 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 18:37:31.500409 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:37:31.500541 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.496525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:36997 (local address 127.18.125.130:57213)
0806 18:37:31.496811 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:31.496829 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:31.496946 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:31.497300 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:31.497303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:31.497331 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:31.497520 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.497530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.498354 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.498360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.499237 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.499248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.499985 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.499991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.500157 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.500185 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:37:31.500278 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:37:31.500350 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":9}
I20260806 18:37:31.500595 1503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.496547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:43067 (local address 127.18.125.130:53365)
0806 18:37:31.497083 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:31.497097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:31.497195 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:31.497466 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:31.497468 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:31.497483 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:31.497641 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.497650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.498447 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.498450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.499330 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:31.499346 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.500095 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.500098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:31.500286 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.500314 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:37:31.500391 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:37:31.500460 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":64,"thread_start_us":152,"threads_started":1}
I20260806 18:37:31.500651 1492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.497033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:53365 (local address 127.18.125.129:43067)
0806 18:37:31.497174 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:31.497178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:31.497199 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:37:31.497301 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:31.497304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:31.497357 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:31.497448 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:31.497773 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.498313 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.499485 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.499969 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.500105 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.500195 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:31.500432 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 18:37:31.500515 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:37:31.501359 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3541eaa45e8940b4847c4c49e517c2d5" candidate_uuid: "5498db8406b44b3aa0657eeb79e81479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43982f4af9314e91b1e68e0f20ad74c9" is_pre_election: true
I20260806 18:37:31.501360 1431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3541eaa45e8940b4847c4c49e517c2d5" candidate_uuid: "5498db8406b44b3aa0657eeb79e81479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8920a68a635465ba34d05579edde726" is_pre_election: true
I20260806 18:37:31.501704 1153 raft_consensus.cc:2454] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498db8406b44b3aa0657eeb79e81479 in term 0.
I20260806 18:37:31.501710 1431 raft_consensus.cc:2454] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5498db8406b44b3aa0657eeb79e81479 in term 0.
I20260806 18:37:31.502269 1223 leader_election.cc:302] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43982f4af9314e91b1e68e0f20ad74c9, 5498db8406b44b3aa0657eeb79e81479; no voters:
I20260806 18:37:31.502579 1502 raft_consensus.cc:2781] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:37:31.502698 1502 raft_consensus.cc:482] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:37:31.502758 1502 raft_consensus.cc:3037] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:31.503929 1502 raft_consensus.cc:504] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } }
I20260806 18:37:31.504339 1502 leader_election.cc:288] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [CANDIDATE]: Term 1 election: Requested vote from peers f8920a68a635465ba34d05579edde726 (127.18.125.131:36997), 43982f4af9314e91b1e68e0f20ad74c9 (127.18.125.129:43067)
I20260806 18:37:31.504760 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3541eaa45e8940b4847c4c49e517c2d5" candidate_uuid: "5498db8406b44b3aa0657eeb79e81479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43982f4af9314e91b1e68e0f20ad74c9"
I20260806 18:37:31.504755 1431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3541eaa45e8940b4847c4c49e517c2d5" candidate_uuid: "5498db8406b44b3aa0657eeb79e81479" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8920a68a635465ba34d05579edde726"
I20260806 18:37:31.504878 1153 raft_consensus.cc:3037] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:31.504911 1431 raft_consensus.cc:3037] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:31.506127 1153 raft_consensus.cc:2454] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498db8406b44b3aa0657eeb79e81479 in term 1.
I20260806 18:37:31.506161 1431 raft_consensus.cc:2454] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5498db8406b44b3aa0657eeb79e81479 in term 1.
I20260806 18:37:31.506443 1223 leader_election.cc:302] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43982f4af9314e91b1e68e0f20ad74c9, 5498db8406b44b3aa0657eeb79e81479; no voters:
I20260806 18:37:31.506654 1502 raft_consensus.cc:2781] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:31.506978 1502 raft_consensus.cc:686] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 1 LEADER]: Becoming Leader. State: Replica: 5498db8406b44b3aa0657eeb79e81479, State: Running, Role: LEADER
I20260806 18:37:31.507365 1502 consensus_queue.cc:237] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } }
I20260806 18:37:31.511104 1001 catalog_manager.cc:5495] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130). New cstate: current_term: 1 leader_uuid: "5498db8406b44b3aa0657eeb79e81479" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } health_report { overall_health: HEALTHY } } }
I20260806 18:37:31.542124 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.538137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36784 (local address 127.18.125.130:41465)
0806 18:37:31.538320 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:31.538325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:31.538343 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:37:31.538444 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:31.538447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:31.538508 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:31.538632 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:37:31.538926 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.539451 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.540446 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:31.540673 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:31.540791 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:31.541158 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:37:31.541632 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:37:31.541778 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:31.541853 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 18:37:31.541924 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:37:31.549352 1153 raft_consensus.cc:1261] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Refusing update from remote peer 5498db8406b44b3aa0657eeb79e81479: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:31.549376 1431 raft_consensus.cc:1261] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726 [term 1 FOLLOWER]: Refusing update from remote peer 5498db8406b44b3aa0657eeb79e81479: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:31.550181 1504 consensus_queue.cc:1035] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:31.550411 1502 consensus_queue.cc:1035] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:31.569269 1001 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:31.569522 1001 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:31.572650 1001 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Sending DeleteTablet for 3 replicas of tablet 3541eaa45e8940b4847c4c49e517c2d5
I20260806 18:37:31.573776 1271 tablet_service.cc:1514] Processing DeleteTablet for tablet 3541eaa45e8940b4847c4c49e517c2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:31 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:36770
I20260806 18:37:31.573845 1410 tablet_service.cc:1514] Processing DeleteTablet for tablet 3541eaa45e8940b4847c4c49e517c2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:31 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:50832
I20260806 18:37:31.573838 1133 tablet_service.cc:1514] Processing DeleteTablet for tablet 3541eaa45e8940b4847c4c49e517c2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:31 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:54520
I20260806 18:37:31.573957 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 1070
I20260806 18:37:31.575716 1517 tablet_replica.cc:330] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726: stopping tablet replica
I20260806 18:37:31.576200 1517 raft_consensus.cc:2229] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:37:31.576614 1518 tablet_replica.cc:330] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: stopping tablet replica
I20260806 18:37:31.577170 1518 raft_consensus.cc:2229] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:37:31.577721 1517 raft_consensus.cc:2258] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:31.578476 1518 raft_consensus.cc:2258] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:31.580263 1518 ts_tablet_manager.cc:1905] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:31.581331 1517 ts_tablet_manager.cc:1905] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:31.583966 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 1206
W20260806 18:37:31.584074 991 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:43067 (error 108)
I20260806 18:37:31.587507 1517 ts_tablet_manager.cc:1918] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:31.587664 1517 log.cc:1196] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/3541eaa45e8940b4847c4c49e517c2d5
I20260806 18:37:31.588024 1517 ts_tablet_manager.cc:1939] T 3541eaa45e8940b4847c4c49e517c2d5 P f8920a68a635465ba34d05579edde726: Deleting consensus metadata
I20260806 18:37:31.588526 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.586263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:43067 (local address 127.0.0.1:54526)
0806 18:37:31.588368 (+ 2105us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:31.588450 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:43067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2007}
W20260806 18:37:31.588795 991 catalog_manager.cc:4584] TS 43982f4af9314e91b1e68e0f20ad74c9 (127.18.125.129:43067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3541eaa45e8940b4847c4c49e517c2d5: Network error: Client connection negotiation failed: client connection to 127.18.125.129:43067: connect: Connection refused (error 111)
I20260806 18:37:31.590124 990 catalog_manager.cc:4841] TS f8920a68a635465ba34d05579edde726 (127.18.125.131:36997): tablet 3541eaa45e8940b4847c4c49e517c2d5 (table test-table [id=e9e0ff4dca5842da96f857dce1f27f1b]) successfully deleted
I20260806 18:37:31.594321 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 1345
I20260806 18:37:31.598399 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:31.598036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:41465 (local address 127.0.0.1:36800)
0806 18:37:31.598223 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:31.598301 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.130:41465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 18:37:31.603477 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 974
I20260806 18:37:31.612610 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34987
--webserver_interface=127.18.125.190
--webserver_port=35535
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:34987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:31.731220 1520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:31.731603 1520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:31.731690 1520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:31.736047 1520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:31.736191 1520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:31.736231 1520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:31.736290 1520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:31.736347 1520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:31.742146 1520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:34987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:31.743345 1520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:31.744815 1520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:31.763796 1527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.764071 1526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.763796 1530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:31.764528 1520 server_base.cc:1034] running on GCE node
I20260806 18:37:31.765403 1520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:31.767259 1520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:31.768464 1520 hybrid_clock.cc:648] HybridClock initialized: now 1786041451768443 us; error 44 us; skew 500 ppm
Aug 06 18:37:31 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:31.772766 1520 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:37:31.773988 1520 webserver.cc:460] Webserver started at http://127.18.125.190:35535/ using document root <none> and password file <none>
I20260806 18:37:31.774767 1520 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:31.774875 1520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:31.780742 1520 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:37:31.783624 1537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:31.785223 1520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:31.785414 1520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5"
format_stamp: "Formatted at 2026-08-06 18:37:30 on dist-test-slave-8w9v"
server_key: "5507aadaec10b2cd51a5f9cba60470ed"
server_key_iv: "f92faa77fb1da25754e8a8a2e8962c56"
server_key_version: "encryptionkey@0"
I20260806 18:37:31.786156 1520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:31.800575 1520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:31.803853 1520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:31.804093 1520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:31.812983 1520 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:34987
I20260806 18:37:31.813297 1592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:34987 every 8 connection(s)
I20260806 18:37:31.814314 1520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:37:31.819929 1593 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:37:31.821195 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 1520
I20260806 18:37:31.821659 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:43067
--local_ip_for_outbound_sockets=127.18.125.129
--tserver_master_addrs=127.18.125.190:34987
--webserver_port=38809
--webserver_interface=127.18.125.129
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:37:31.822119 1593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Bootstrap starting.
I20260806 18:37:31.840763 1593 log.cc:824] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:31.850674 1593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:37:31.851739 1593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Bootstrap complete.
I20260806 18:37:31.856570 1593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:31.857085 1593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56d6d2a8b96b45a1a1068d39f55f1dc5, State: Initialized, Role: FOLLOWER
I20260806 18:37:31.857676 1593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:31.857838 1593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:37:31.857936 1593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:37:31.858053 1593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:37:31.860312 1593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:31.860678 1593 leader_election.cc:302] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56d6d2a8b96b45a1a1068d39f55f1dc5; no voters:
I20260806 18:37:31.861060 1593 leader_election.cc:288] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:37:31.861159 1597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:37:31.861353 1597 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 2 LEADER]: Becoming Leader. State: Replica: 56d6d2a8b96b45a1a1068d39f55f1dc5, State: Running, Role: LEADER
I20260806 18:37:31.861665 1597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:31.862440 1593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:31.863106 1598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } } }
I20260806 18:37:31.863237 1598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:31.863577 1599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56d6d2a8b96b45a1a1068d39f55f1dc5. Latest consensus state: current_term: 2 leader_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } } }
I20260806 18:37:31.863672 1599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:31.864301 1602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:37:31.869678 1602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:37:31.870512 1602 catalog_manager.cc:1259] Loaded cluster ID: fab6bf16aae84f929efc66dad8eff566
I20260806 18:37:31.870590 1602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:37:31.872470 1602 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 18:37:31.876947 1602 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 18:37:31.883219 1602 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:37:31.884228 1602 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Loaded TSK: 0
I20260806 18:37:31.885620 1602 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:37:31.952512 1595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:31.952795 1595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:31.952839 1595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:31.957520 1595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:31.957657 1595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:31.957784 1595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:37:31.964049 1595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:43067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=38809
--tserver_master_addrs=127.18.125.190:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:31.965401 1595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:31.966548 1595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:31.977699 1620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.977878 1623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:31.977720 1621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:31.978762 1595 server_base.cc:1034] running on GCE node
I20260806 18:37:31.979266 1595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:31.980002 1595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:31.981240 1595 hybrid_clock.cc:648] HybridClock initialized: now 1786041451981216 us; error 46 us; skew 500 ppm
Aug 06 18:37:31 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:31.984591 1595 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:37:31.985721 1595 webserver.cc:460] Webserver started at http://127.18.125.129:38809/ using document root <none> and password file <none>
I20260806 18:37:31.986366 1595 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:31.986491 1595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:31.991014 1595 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:37:31.993666 1630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:31.995129 1595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:37:31.995266 1595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "43982f4af9314e91b1e68e0f20ad74c9"
format_stamp: "Formatted at 2026-08-06 18:37:30 on dist-test-slave-8w9v"
server_key: "ba7bba57a2da97ec48ece151de8244a0"
server_key_iv: "6e872a69fe6356a35aa18173e6664e39"
server_key_version: "encryptionkey@0"
I20260806 18:37:31.995879 1595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:32.008352 1595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:32.011902 1595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:32.012089 1595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:32.012867 1595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:32.014231 1638 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:37:32.019006 1595 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:37:32.019093 1595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 18:37:32.019151 1595 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:37:32.021193 1595 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:37:32.021265 1595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 18:37:32.021545 1638 tablet_bootstrap.cc:492] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Bootstrap starting.
I20260806 18:37:32.033017 1595 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:43067
I20260806 18:37:32.033238 1746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:43067 every 8 connection(s)
I20260806 18:37:32.034539 1595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:37:32.039882 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 1595
I20260806 18:37:32.040522 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:41465
--local_ip_for_outbound_sockets=127.18.125.130
--tserver_master_addrs=127.18.125.190:34987
--webserver_port=34285
--webserver_interface=127.18.125.130
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:37:32.045913 1638 log.cc:824] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:37:32 dist-test-slave-8w9v krb5kdc[961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:32.055572 1638 tablet_bootstrap.cc:492] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:37:32.056083 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.037097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:36755 (local address 127.18.125.190:34987)
0806 18:37:32.037570 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.037586 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.038162 (+ 576us) server_negotiation.cc:407] Connection header received
0806 18:37:32.040124 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.040191 (+ 67us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.040624 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.041061 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:32.042392 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.043798 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.045168 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.045656 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.045842 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.049226 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:32.049234 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:32.049252 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:32.049291 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:32.051693 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.052481 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.052490 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.052497 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.052711 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.053238 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.053243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.053246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.053850 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:32.054018 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.054710 (+ 692us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.055126 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":138,"threads_started":1}
I20260806 18:37:32.056111 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.037133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:36755)
0806 18:37:32.037928 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.037999 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.039549 (+ 1550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.041120 (+ 1571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.041129 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.041599 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:32.042092 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.042167 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.043969 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.043973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.045021 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.045033 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.045830 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.045834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.046043 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.046765 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:32.046792 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:32.048951 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:32.051880 (+ 2929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.051890 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.051905 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.052317 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.052884 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.052887 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.052889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.053094 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.054002 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:32.054009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:32.054300 (+ 291us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.054504 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.054875 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":245,"threads_started":1}
I20260806 18:37:32.056110 1638 tablet_bootstrap.cc:492] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Bootstrap complete.
I20260806 18:37:32.058125 1638 ts_tablet_manager.cc:1397] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Time spent bootstrapping tablet: real 0.037s user 0.030s sys 0.004s
I20260806 18:37:32.059412 1747 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34987
I20260806 18:37:32.059703 1747 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:32.060387 1747 heartbeater.cc:511] Master 127.18.125.190:34987 requested a full tablet report, sending...
I20260806 18:37:32.061228 1638 raft_consensus.cc:348] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } }
I20260806 18:37:32.061722 1638 raft_consensus.cc:729] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43982f4af9314e91b1e68e0f20ad74c9, State: Initialized, Role: FOLLOWER
I20260806 18:37:32.062232 1638 consensus_queue.cc:260] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } }
I20260806 18:37:32.063345 1638 ts_tablet_manager.cc:1428] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:37:32.063668 1553 ts_manager.cc:194] Registered new tserver with Master: 43982f4af9314e91b1e68e0f20ad74c9 (127.18.125.129:43067)
I20260806 18:37:32.069278 1553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:36755
I20260806 18:37:32.072633 1747 heartbeater.cc:503] Master 127.18.125.190:34987 was elected leader, sending a full tablet report...
Aug 06 18:37:32 dist-test-slave-8w9v krb5kdc[961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:37:32.079668 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.067889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:43067 (local address 127.0.0.1:54536)
0806 18:37:32.068504 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.068562 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.068737 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.069378 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.069384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.069831 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:32.069946 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.069954 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.070976 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.070982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.071652 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.071662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.072395 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.072399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.072528 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.073188 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:32.073203 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:32.074955 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:32.077569 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.077579 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.077584 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.077836 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.078229 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.078231 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.078232 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.078327 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.079045 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:32.079051 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:32.079172 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.079309 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.079519 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":148,"threads_started":1}
I20260806 18:37:32.079838 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.067960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54536 (local address 127.18.125.129:43067)
0806 18:37:32.068884 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.068897 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.068921 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:37:32.069002 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.069006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.069183 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.069360 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:32.070091 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.070830 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.071797 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.072255 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.072384 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.075128 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:32.075141 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:32.075156 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:32.075191 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:32.077249 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.077980 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.077987 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.077990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.078107 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.078447 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.078454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.078457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.078882 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:32.079045 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.079436 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.079643 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":784,"thread_start_us":178,"threads_started":1}
I20260806 18:37:32.081140 1680 tablet_service.cc:1514] Processing DeleteTablet for tablet 3541eaa45e8940b4847c4c49e517c2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:31 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:54536
I20260806 18:37:32.082176 1756 tablet_replica.cc:330] stopping tablet replica
I20260806 18:37:32.082564 1756 raft_consensus.cc:2229] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:37:32.082864 1756 raft_consensus.cc:2258] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:32.084182 1756 ts_tablet_manager.cc:1905] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:32.089702 1756 ts_tablet_manager.cc:1918] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:32.089851 1756 log.cc:1196] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal/wals/3541eaa45e8940b4847c4c49e517c2d5
I20260806 18:37:32.090176 1756 ts_tablet_manager.cc:1939] T 3541eaa45e8940b4847c4c49e517c2d5 P 43982f4af9314e91b1e68e0f20ad74c9: Deleting consensus metadata
I20260806 18:37:32.092597 1539 catalog_manager.cc:4841] TS 43982f4af9314e91b1e68e0f20ad74c9 (127.18.125.129:43067): tablet 3541eaa45e8940b4847c4c49e517c2d5 (table test-table [id=e9e0ff4dca5842da96f857dce1f27f1b]) successfully deleted
W20260806 18:37:32.179636 1751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:32.179978 1751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:32.180029 1751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:32.184327 1751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:32.184410 1751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:32.184513 1751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:37:32.190367 1751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:41465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=34285
--tserver_master_addrs=127.18.125.190:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:32.191730 1751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:32.192903 1751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:32.203518 1761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:32.203614 1751 server_base.cc:1034] running on GCE node
W20260806 18:37:32.203522 1762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:32.203518 1764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:32.204294 1751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:32.205006 1751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:32.206194 1751 hybrid_clock.cc:648] HybridClock initialized: now 1786041452206170 us; error 38 us; skew 500 ppm
Aug 06 18:37:32 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041452, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:32.209240 1751 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:37:32.210110 1751 webserver.cc:460] Webserver started at http://127.18.125.130:34285/ using document root <none> and password file <none>
I20260806 18:37:32.210644 1751 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:32.210716 1751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:32.214624 1751 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:37:32.216864 1771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:32.218292 1751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:32.218477 1751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "5498db8406b44b3aa0657eeb79e81479"
format_stamp: "Formatted at 2026-08-06 18:37:31 on dist-test-slave-8w9v"
server_key: "91cb82edd7cee5b38247dfb6aba9f37b"
server_key_iv: "a0bcdc75999954ae770a70f8c029b2ed"
server_key_version: "encryptionkey@0"
I20260806 18:37:32.219050 1751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:32.241859 1751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:32.245847 1751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:32.246119 1751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:32.246855 1751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:32.248378 1779 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260806 18:37:32.254334 1779 ts_tablet_manager.cc:1852] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 18:37:32.254925 1779 ts_tablet_manager.cc:1905] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:32.259969 1779 ts_tablet_manager.cc:1918] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:32.260219 1779 log.cc:1196] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/3541eaa45e8940b4847c4c49e517c2d5
I20260806 18:37:32.260651 1779 ts_tablet_manager.cc:1939] T 3541eaa45e8940b4847c4c49e517c2d5 P 5498db8406b44b3aa0657eeb79e81479: Deleting consensus metadata
I20260806 18:37:32.261092 1751 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 18:37:32.261169 1751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.013s user 0.000s sys 0.000s
I20260806 18:37:32.261273 1751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:32.261346 1751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:32.272817 1751 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:41465
I20260806 18:37:32.272872 1890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:41465 every 8 connection(s)
I20260806 18:37:32.273800 1751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:37:32.283210 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 1751
I20260806 18:37:32.283736 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:36997
--local_ip_for_outbound_sockets=127.18.125.131
--tserver_master_addrs=127.18.125.190:34987
--webserver_port=46177
--webserver_interface=127.18.125.131
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:37:32 dist-test-slave-8w9v krb5kdc[961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041452, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:32.289638 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.275962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:43079 (local address 127.18.125.190:34987)
0806 18:37:32.276143 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.276148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.277123 (+ 975us) server_negotiation.cc:407] Connection header received
0806 18:37:32.278088 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.278092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.278151 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.278232 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:32.279531 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.280099 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.281795 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.282049 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.282156 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.285660 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:32.285664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:32.285667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:32.285706 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:32.287231 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.287821 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.287825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.287826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.287920 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.288372 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.288378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.288383 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.288627 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:32.288742 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.289312 (+ 570us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.289468 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:37:32.290704 1893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.276410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:43079)
0806 18:37:32.276933 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.276994 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.277872 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.278410 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.278416 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.278820 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:32.279343 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.279359 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.280482 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.280555 (+ 73us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.281581 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.281598 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.282469 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.282473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.282653 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.283535 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:32.283562 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:32.285446 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:32.287380 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.287387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.287407 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.287698 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.288020 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.288024 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.288026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.288268 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.288742 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:32.288752 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:32.289049 (+ 297us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.289258 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.289645 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":114,"threads_started":1}
I20260806 18:37:32.292438 1891 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34987
I20260806 18:37:32.292748 1891 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:32.293316 1891 heartbeater.cc:511] Master 127.18.125.190:34987 requested a full tablet report, sending...
I20260806 18:37:32.294713 1553 ts_manager.cc:194] Registered new tserver with Master: 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465)
I20260806 18:37:32.295652 1553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:43079
W20260806 18:37:32.402796 1894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:32.403079 1894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:32.403123 1894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:32.407454 1894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:32.407529 1894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:32.407593 1894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:37:32.413308 1894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:36997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=46177
--tserver_master_addrs=127.18.125.190:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:32.414429 1894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:32.415439 1894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:32.427263 1902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:32.427314 1899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:32.427315 1900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:32.427451 1894 server_base.cc:1034] running on GCE node
I20260806 18:37:32.428014 1894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:32.428751 1894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:32.429948 1894 hybrid_clock.cc:648] HybridClock initialized: now 1786041452429933 us; error 27 us; skew 500 ppm
Aug 06 18:37:32 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041452, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:32.433050 1894 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:37:32.434022 1894 webserver.cc:460] Webserver started at http://127.18.125.131:46177/ using document root <none> and password file <none>
I20260806 18:37:32.434667 1894 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:32.434792 1894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:32.439762 1894 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260806 18:37:32.442451 1909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:32.443912 1894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:37:32.444088 1894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "f8920a68a635465ba34d05579edde726"
format_stamp: "Formatted at 2026-08-06 18:37:31 on dist-test-slave-8w9v"
server_key: "fc6d911e81fc6630059a3a62f8a103ff"
server_key_iv: "2a85b5dba8dcd84d9953aa5a8eb22d4f"
server_key_version: "encryptionkey@0"
I20260806 18:37:32.444868 1894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:32.467322 1894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:32.471117 1894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:32.471366 1894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:32.472077 1894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:32.473061 1894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:32.473141 1894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:32.473217 1894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:32.473266 1894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:32.483080 1894 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:36997
I20260806 18:37:32.483127 2025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:36997 every 8 connection(s)
I20260806 18:37:32.484261 1894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:37:32.492040 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 1894
Aug 06 18:37:32 dist-test-slave-8w9v krb5kdc[961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041452, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:32.499917 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.486309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:56673 (local address 127.18.125.190:34987)
0806 18:37:32.486485 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.486489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.487347 (+ 858us) server_negotiation.cc:407] Connection header received
0806 18:37:32.488930 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.488934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.488997 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.489098 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:32.490380 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.490953 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.492379 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.492732 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.492882 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.495650 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:32.495654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:32.495657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:32.495684 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:32.497543 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.498173 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.498179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.498183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.498275 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.498669 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.498673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.498675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.498867 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:32.498981 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.499541 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.499704 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 18:37:32.501049 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.486675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:56673)
0806 18:37:32.487165 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.487210 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.488678 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.489290 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.489297 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.489663 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:32.490220 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.490235 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.491090 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.491098 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.492219 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.492233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.492912 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.492919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.493146 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.493877 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:32.493894 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:32.495477 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:32.497701 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.497707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.497722 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.498024 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.498390 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:32.498394 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:32.498396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:32.498559 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:32.498981 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:32.498989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:32.499260 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.499495 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.499888 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":117,"threads_started":1}
I20260806 18:37:32.501850 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.492778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41016 (local address 127.18.125.190:34987)
0806 18:37:32.493092 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.493097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.493276 (+ 179us) server_negotiation.cc:407] Connection header received
0806 18:37:32.493376 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.493382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.493440 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.493528 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:32.494347 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.494946 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.496028 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.496388 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.496476 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.498042 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:32.498046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:32.498049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:32.498075 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:32.499599 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.500156 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.500161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.500163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.500244 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:32.500596 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:32.500600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:32.500601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:32.500806 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:32.501062 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.501576 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.501713 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":101,"threads_started":1}
I20260806 18:37:32.502568 2026 heartbeater.cc:347] Connected to a master server at 127.18.125.190:34987
I20260806 18:37:32.502838 2026 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:32.503479 2026 heartbeater.cc:511] Master 127.18.125.190:34987 requested a full tablet report, sending...
I20260806 18:37:32.504624 1553 ts_manager.cc:194] Registered new tserver with Master: f8920a68a635465ba34d05579edde726 (127.18.125.131:36997)
I20260806 18:37:32.505146 1553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:56673
I20260806 18:37:32.515944 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:37:32.521040 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.516737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41022 (local address 127.18.125.190:34987)
0806 18:37:32.516910 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.516916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.517095 (+ 179us) server_negotiation.cc:407] Connection header received
0806 18:37:32.517343 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.517345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.517462 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.517583 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:37:32.517869 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.518416 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.519344 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.519587 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.519709 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.520064 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:37:32.520581 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:37:32.520722 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.520794 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.520869 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:37:32.521833 1553 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:37:32.525657 1553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:37:32.534933 1680 tablet_service.cc:1467] Processing CreateTablet for tablet b753952cc60647d4b2c82089de95ec21 (DEFAULT_TABLE table=test-table [id=f76a550d254144738c3af3e4cb8f629e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:32.535651 1680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b753952cc60647d4b2c82089de95ec21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:32.538834 1638 tablet_bootstrap.cc:492] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9: Bootstrap starting.
I20260806 18:37:32.539661 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.532830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:36997 (local address 127.0.0.1:50846)
0806 18:37:32.533006 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.533022 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.533102 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.534293 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.534355 (+ 62us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.534377 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:32.534548 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.534579 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.536462 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.536466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.537876 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.537886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.539180 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.539184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.539292 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.539350 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.539411 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.539520 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":25}
I20260806 18:37:32.539834 1638 tablet_bootstrap.cc:654] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:32.540275 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.533204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:41465 (local address 127.0.0.1:36808)
0806 18:37:32.533892 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.533907 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.534004 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.534925 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.534927 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.534946 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:32.535045 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.535051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.536823 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.536826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.538441 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.538453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.539557 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.539561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.539678 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.539831 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.539984 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.540070 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260806 18:37:32.540385 2037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.533900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36808 (local address 127.18.125.130:41465)
0806 18:37:32.534320 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.534334 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.534355 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:37:32.534433 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.534441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.534609 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.534765 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:32.535310 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.536651 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.538583 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.539421 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.539545 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.539684 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.540084 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.540213 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":120,"threads_started":1}
I20260806 18:37:32.540544 2036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.533061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50846 (local address 127.18.125.131:36997)
0806 18:37:32.533706 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.533721 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.533746 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:37:32.533838 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.533846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.534092 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.534276 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:32.534777 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.536298 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.537989 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.539031 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.539889 (+ 858us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.540032 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.540081 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.540216 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":236,"threads_started":1}
I20260806 18:37:32.541607 1638 tablet_bootstrap.cc:492] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9: No bootstrap required, opened a new log
I20260806 18:37:32.541702 1638 ts_tablet_manager.cc:1397] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 18:37:32.542171 1638 raft_consensus.cc:348] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:32.542302 1638 raft_consensus.cc:374] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:32.542339 1638 raft_consensus.cc:729] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43982f4af9314e91b1e68e0f20ad74c9, State: Initialized, Role: FOLLOWER
I20260806 18:37:32.542481 1638 consensus_queue.cc:260] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:32.542157 1822 tablet_service.cc:1467] Processing CreateTablet for tablet b753952cc60647d4b2c82089de95ec21 (DEFAULT_TABLE table=test-table [id=f76a550d254144738c3af3e4cb8f629e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:32.542737 1638 ts_tablet_manager.cc:1428] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:37:32.543385 1822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b753952cc60647d4b2c82089de95ec21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:32.543751 1958 tablet_service.cc:1467] Processing CreateTablet for tablet b753952cc60647d4b2c82089de95ec21 (DEFAULT_TABLE table=test-table [id=f76a550d254144738c3af3e4cb8f629e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:32.544838 1958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b753952cc60647d4b2c82089de95ec21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:32.548449 1779 tablet_bootstrap.cc:492] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479: Bootstrap starting.
I20260806 18:37:32.550433 1779 tablet_bootstrap.cc:654] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:32.551183 1779 log.cc:824] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:32.552917 2040 tablet_bootstrap.cc:492] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726: Bootstrap starting.
I20260806 18:37:32.553342 1779 tablet_bootstrap.cc:492] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479: No bootstrap required, opened a new log
I20260806 18:37:32.553526 1779 ts_tablet_manager.cc:1397] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260806 18:37:32.555202 2040 tablet_bootstrap.cc:654] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:32.556597 2040 log.cc:824] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:32.557739 1779 raft_consensus.cc:348] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:32.558010 1779 raft_consensus.cc:374] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:32.558065 1779 raft_consensus.cc:729] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498db8406b44b3aa0657eeb79e81479, State: Initialized, Role: FOLLOWER
I20260806 18:37:32.558555 1779 consensus_queue.cc:260] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:32.558847 2040 tablet_bootstrap.cc:492] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726: No bootstrap required, opened a new log
I20260806 18:37:32.559057 2040 ts_tablet_manager.cc:1397] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:37:32.559327 1891 heartbeater.cc:503] Master 127.18.125.190:34987 was elected leader, sending a full tablet report...
I20260806 18:37:32.560694 1779 ts_tablet_manager.cc:1428] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 18:37:32.563122 2040 raft_consensus.cc:348] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:32.563437 2040 raft_consensus.cc:374] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:32.563521 2040 raft_consensus.cc:729] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8920a68a635465ba34d05579edde726, State: Initialized, Role: FOLLOWER
I20260806 18:37:32.563936 2040 consensus_queue.cc:260] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:32.564661 2026 heartbeater.cc:503] Master 127.18.125.190:34987 was elected leader, sending a full tablet report...
I20260806 18:37:32.565018 2040 ts_tablet_manager.cc:1428] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260806 18:37:32.633723 2043 raft_consensus.cc:482] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:32.633951 2043 raft_consensus.cc:504] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:32.635159 2043 leader_election.cc:288] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465), 43982f4af9314e91b1e68e0f20ad74c9 (127.18.125.129:43067)
I20260806 18:37:32.639792 2037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.635576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:55385 (local address 127.18.125.130:41465)
0806 18:37:32.635734 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.635739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.635757 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:37:32.635823 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.635826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.635880 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.636020 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:32.636451 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.637151 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.638453 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.639159 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.639315 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.639387 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.639588 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.639662 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:37:32.639791 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.635446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:41465 (local address 127.18.125.131:55385)
0806 18:37:32.635568 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.635585 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.635696 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.636138 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.636142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.636168 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:32.636323 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.636332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.637292 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.637295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.638292 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.638303 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.639316 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.639320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.639437 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.639464 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.639557 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.639631 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 18:37:32.640564 2044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.635756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:43067 (local address 127.18.125.131:40351)
0806 18:37:32.636143 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.636156 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:32.636245 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:32.636708 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:32.636710 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:32.636726 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:32.636863 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.636871 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.637884 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.637887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.639119 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:32.639134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.640158 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.640162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:32.640267 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.640288 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:37:32.640358 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:37:32.640423 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":139,"threads_started":1}
I20260806 18:37:32.640576 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.635881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:40351 (local address 127.18.125.129:43067)
0806 18:37:32.636193 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.636197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.636212 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:37:32.636371 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.636376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.636479 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.636599 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:32.637000 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.637731 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.639322 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.639994 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.640142 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.640323 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.640372 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.640451 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":103,"threads_started":1}
I20260806 18:37:32.640527 1843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b753952cc60647d4b2c82089de95ec21" candidate_uuid: "f8920a68a635465ba34d05579edde726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498db8406b44b3aa0657eeb79e81479" is_pre_election: true
I20260806 18:37:32.640933 1843 raft_consensus.cc:2454] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8920a68a635465ba34d05579edde726 in term 0.
I20260806 18:37:32.641104 1700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b753952cc60647d4b2c82089de95ec21" candidate_uuid: "f8920a68a635465ba34d05579edde726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43982f4af9314e91b1e68e0f20ad74c9" is_pre_election: true
I20260806 18:37:32.641470 1700 raft_consensus.cc:2454] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8920a68a635465ba34d05579edde726 in term 0.
I20260806 18:37:32.641515 1911 leader_election.cc:302] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498db8406b44b3aa0657eeb79e81479, f8920a68a635465ba34d05579edde726; no voters:
I20260806 18:37:32.641812 2043 raft_consensus.cc:2781] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:37:32.641906 2043 raft_consensus.cc:482] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:37:32.641952 2043 raft_consensus.cc:3037] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:32.643294 2043 raft_consensus.cc:504] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:32.643749 2043 leader_election.cc:288] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [CANDIDATE]: Term 1 election: Requested vote from peers 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465), 43982f4af9314e91b1e68e0f20ad74c9 (127.18.125.129:43067)
I20260806 18:37:32.644212 1843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b753952cc60647d4b2c82089de95ec21" candidate_uuid: "f8920a68a635465ba34d05579edde726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5498db8406b44b3aa0657eeb79e81479"
I20260806 18:37:32.644239 1700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b753952cc60647d4b2c82089de95ec21" candidate_uuid: "f8920a68a635465ba34d05579edde726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43982f4af9314e91b1e68e0f20ad74c9"
I20260806 18:37:32.644363 1843 raft_consensus.cc:3037] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:32.644393 1700 raft_consensus.cc:3037] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:32.645476 1843 raft_consensus.cc:2454] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8920a68a635465ba34d05579edde726 in term 1.
I20260806 18:37:32.645511 1700 raft_consensus.cc:2454] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8920a68a635465ba34d05579edde726 in term 1.
I20260806 18:37:32.645795 1911 leader_election.cc:302] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5498db8406b44b3aa0657eeb79e81479, f8920a68a635465ba34d05579edde726; no voters:
I20260806 18:37:32.646036 2043 raft_consensus.cc:2781] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:32.646338 2043 raft_consensus.cc:686] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 1 LEADER]: Becoming Leader. State: Replica: f8920a68a635465ba34d05579edde726, State: Running, Role: LEADER
I20260806 18:37:32.646730 2043 consensus_queue.cc:237] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:32.650421 1553 catalog_manager.cc:5495] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8920a68a635465ba34d05579edde726 (127.18.125.131). New cstate: current_term: 1 leader_uuid: "f8920a68a635465ba34d05579edde726" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } health_report { overall_health: UNKNOWN } } }
I20260806 18:37:32.670492 2036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.666993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50854 (local address 127.18.125.131:36997)
0806 18:37:32.667157 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:32.667162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:32.667179 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:37:32.667276 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:32.667279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:32.667343 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:32.667467 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:37:32.667683 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.668345 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.669152 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:32.669388 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:32.669479 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:32.669734 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:37:32.670175 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:37:32.670277 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:32.670324 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 18:37:32.670376 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:37:32.676853 1843 raft_consensus.cc:1261] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479 [term 1 FOLLOWER]: Refusing update from remote peer f8920a68a635465ba34d05579edde726: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:32.676856 1700 raft_consensus.cc:1261] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Refusing update from remote peer f8920a68a635465ba34d05579edde726: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:32.677747 2046 consensus_queue.cc:1035] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:32.677973 2043 consensus_queue.cc:1035] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:32.686066 2050 mvcc.cc:205] Tried to move back new op lower bound from 7315625790156099584 to 7315625790044057600. Current Snapshot: MvccSnapshot[applied={T|T < 7315625790156099584}]
I20260806 18:37:32.685981 2051 mvcc.cc:205] Tried to move back new op lower bound from 7315625790156099584 to 7315625790044057600. Current Snapshot: MvccSnapshot[applied={T|T < 7315625790156099584}]
I20260806 18:37:32.694504 1555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:32.694689 1555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:32.697510 1555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Sending DeleteTablet for 3 replicas of tablet b753952cc60647d4b2c82089de95ec21
I20260806 18:37:32.698269 1958 tablet_service.cc:1514] Processing DeleteTablet for tablet b753952cc60647d4b2c82089de95ec21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:32 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:50846
I20260806 18:37:32.698453 1680 tablet_service.cc:1514] Processing DeleteTablet for tablet b753952cc60647d4b2c82089de95ec21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:32 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:54536
I20260806 18:37:32.698558 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 1595
I20260806 18:37:32.698514 1822 tablet_service.cc:1514] Processing DeleteTablet for tablet b753952cc60647d4b2c82089de95ec21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:32 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:36808
I20260806 18:37:32.699728 2060 tablet_replica.cc:330] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479: stopping tablet replica
I20260806 18:37:32.700258 2060 raft_consensus.cc:2229] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:37:32.700757 2060 raft_consensus.cc:2258] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:32.701573 2059 tablet_replica.cc:330] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726: stopping tablet replica
I20260806 18:37:32.701861 2059 raft_consensus.cc:2229] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:37:32.702299 2059 raft_consensus.cc:2258] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:32.704346 2060 ts_tablet_manager.cc:1905] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:32.704579 2059 ts_tablet_manager.cc:1905] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:32.708369 2060 ts_tablet_manager.cc:1918] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:32.708369 2059 ts_tablet_manager.cc:1918] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:32.708534 2059 log.cc:1196] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/b753952cc60647d4b2c82089de95ec21
I20260806 18:37:32.708725 2060 log.cc:1196] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/b753952cc60647d4b2c82089de95ec21
I20260806 18:37:32.708999 2059 ts_tablet_manager.cc:1939] T b753952cc60647d4b2c82089de95ec21 P f8920a68a635465ba34d05579edde726: Deleting consensus metadata
I20260806 18:37:32.709232 2060 ts_tablet_manager.cc:1939] T b753952cc60647d4b2c82089de95ec21 P 5498db8406b44b3aa0657eeb79e81479: Deleting consensus metadata
W20260806 18:37:32.709965 1539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:43067 (error 108)
I20260806 18:37:32.710625 1538 catalog_manager.cc:4841] TS f8920a68a635465ba34d05579edde726 (127.18.125.131:36997): tablet b753952cc60647d4b2c82089de95ec21 (table test-table [id=f76a550d254144738c3af3e4cb8f629e]) successfully deleted
I20260806 18:37:32.710625 1539 catalog_manager.cc:4841] TS 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465): tablet b753952cc60647d4b2c82089de95ec21 (table test-table [id=f76a550d254144738c3af3e4cb8f629e]) successfully deleted
I20260806 18:37:32.711261 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 1751
I20260806 18:37:32.717284 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:32.716922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:43067 (local address 127.0.0.1:54544)
0806 18:37:32.717094 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:32.717178 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:43067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 18:37:32.717609 1539 catalog_manager.cc:4584] TS 43982f4af9314e91b1e68e0f20ad74c9 (127.18.125.129:43067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b753952cc60647d4b2c82089de95ec21: Network error: Client connection negotiation failed: client connection to 127.18.125.129:43067: connect: Connection refused (error 111)
I20260806 18:37:32.720084 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 1894
I20260806 18:37:32.728439 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 1520
I20260806 18:37:32.737212 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34987
--webserver_interface=127.18.125.190
--webserver_port=35535
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:34987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:32.882189 2062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:32.882514 2062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:32.882561 2062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:32.886847 2062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:32.886938 2062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:32.886962 2062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:32.886983 2062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:32.887034 2062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:32.893023 2062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:34987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=35535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:32.894290 2062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:32.895462 2062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:32.905452 2068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:32.905453 2070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:32.905560 2062 server_base.cc:1034] running on GCE node
W20260806 18:37:32.905452 2067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:32.906629 2062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:32.907967 2062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:32.909207 2062 hybrid_clock.cc:648] HybridClock initialized: now 1786041452909188 us; error 36 us; skew 500 ppm
Aug 06 18:37:32 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041452, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:32.912407 2062 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:37:32.913482 2062 webserver.cc:460] Webserver started at http://127.18.125.190:35535/ using document root <none> and password file <none>
I20260806 18:37:32.914121 2062 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:32.914228 2062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:32.919116 2062 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 18:37:32.921836 2078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:32.923386 2062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:37:32.923508 2062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5"
format_stamp: "Formatted at 2026-08-06 18:37:30 on dist-test-slave-8w9v"
server_key: "5507aadaec10b2cd51a5f9cba60470ed"
server_key_iv: "f92faa77fb1da25754e8a8a2e8962c56"
server_key_version: "encryptionkey@0"
I20260806 18:37:32.924095 2062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:32.935781 2062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:32.939320 2062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:32.939559 2062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:32.947311 2062 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:34987
I20260806 18:37:32.947645 2131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:34987 every 8 connection(s)
I20260806 18:37:32.948606 2062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:37:32.953516 2132 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:37:32.955108 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 2062
I20260806 18:37:32.955651 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:43067
--local_ip_for_outbound_sockets=127.18.125.129
--tserver_master_addrs=127.18.125.190:34987
--webserver_port=38809
--webserver_interface=127.18.125.129
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:37:32.956023 2132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Bootstrap starting.
I20260806 18:37:32.977617 2132 log.cc:824] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:32.992848 2132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:37:32.993336 2132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: Bootstrap complete.
I20260806 18:37:32.997709 2132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:32.998106 2132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56d6d2a8b96b45a1a1068d39f55f1dc5, State: Initialized, Role: FOLLOWER
I20260806 18:37:32.998680 2132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:32.998811 2132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:37:32.998888 2132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:37:32.998989 2132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:37:33.001197 2132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:33.001573 2132 leader_election.cc:302] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56d6d2a8b96b45a1a1068d39f55f1dc5; no voters:
I20260806 18:37:33.001998 2132 leader_election.cc:288] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 18:37:33.002161 2137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:37:33.002430 2137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [term 3 LEADER]: Becoming Leader. State: Replica: 56d6d2a8b96b45a1a1068d39f55f1dc5, State: Running, Role: LEADER
I20260806 18:37:33.002767 2137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } }
I20260806 18:37:33.003302 2132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:33.004611 2138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } } }
I20260806 18:37:33.004750 2138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:33.004981 2139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56d6d2a8b96b45a1a1068d39f55f1dc5. Latest consensus state: current_term: 3 leader_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6d2a8b96b45a1a1068d39f55f1dc5" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34987 } } }
I20260806 18:37:33.005069 2139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:33.009305 2153 catalog_manager.cc:1259] Loaded cluster ID: fab6bf16aae84f929efc66dad8eff566
I20260806 18:37:33.009372 2153 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: loading cluster ID for follower catalog manager: success
W20260806 18:37:33.010560 2153 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 56d6d2a8b96b45a1a1068d39f55f1dc5: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260806 18:37:33.010634 2153 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260806 18:37:33.010695 2146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:37:33.014549 2146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:37:33.015120 2146 catalog_manager.cc:1259] Loaded cluster ID: fab6bf16aae84f929efc66dad8eff566
I20260806 18:37:33.015188 2146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 18:37:33.015970 2146 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f48c82dd9ad google::LogMessage::Fail() at ??:0
@ 0x7f48c82e088a google::LogMessage::SendToLog() at ??:0
@ 0x7f48c82dd4de google::LogMessage::Flush() at ??:0
@ 0x7f48c82dea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f48d2c1cfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f48d2c1dd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f48d2c1a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f48d2c5e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f48ca52f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f48ca52fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f48ca5313f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f48ca520512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f48cbb646db start_thread at ??:0
@ 0x7f48c6ddb61f clone at ??:0
W20260806 18:37:33.112929 2135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:33.113227 2135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:33.113273 2135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:33.117604 2135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:33.117666 2135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:33.117728 2135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:37:33.123286 2135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:43067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=38809
--tserver_master_addrs=127.18.125.190:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:33.124534 2135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:33.125629 2135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:33.135713 2176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:33.135713 2179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:33.135707 2177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:33.136245 2135 server_base.cc:1034] running on GCE node
I20260806 18:37:33.136848 2135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:33.137737 2135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:33.138959 2135 hybrid_clock.cc:648] HybridClock initialized: now 1786041453138918 us; error 49 us; skew 500 ppm
Aug 06 18:37:33 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041453, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:33.142441 2135 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:37:33.143648 2135 webserver.cc:460] Webserver started at http://127.18.125.129:38809/ using document root <none> and password file <none>
I20260806 18:37:33.144384 2135 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:33.144462 2135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:33.148708 2135 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:33.151141 2186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:33.152608 2135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:33.152761 2135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "43982f4af9314e91b1e68e0f20ad74c9"
format_stamp: "Formatted at 2026-08-06 18:37:30 on dist-test-slave-8w9v"
server_key: "ba7bba57a2da97ec48ece151de8244a0"
server_key_iv: "6e872a69fe6356a35aa18173e6664e39"
server_key_version: "encryptionkey@0"
I20260806 18:37:33.153364 2135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:33.171841 2135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:33.175836 2135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:33.176069 2135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:33.176721 2135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:33.177937 2193 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:37:33.183650 2135 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:37:33.183763 2135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 18:37:33.183880 2135 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:37:33.185699 2135 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:37:33.185773 2135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:37:33.186098 2193 tablet_bootstrap.cc:492] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9: Bootstrap starting.
I20260806 18:37:33.197194 2135 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:43067
I20260806 18:37:33.197944 2301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:43067 every 8 connection(s)
I20260806 18:37:33.198418 2135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:37:33.204610 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 2135
I20260806 18:37:33.205173 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:41465
--local_ip_for_outbound_sockets=127.18.125.130
--tserver_master_addrs=127.18.125.190:34987
--webserver_port=34285
--webserver_interface=127.18.125.130
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:37:33.211143 2193 log.cc:824] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:33.220700 2193 tablet_bootstrap.cc:492] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:37:33.221190 2193 tablet_bootstrap.cc:492] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9: Bootstrap complete.
I20260806 18:37:33.222755 2193 ts_tablet_manager.cc:1397] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9: Time spent bootstrapping tablet: real 0.037s user 0.024s sys 0.009s
I20260806 18:37:33.226162 2193 raft_consensus.cc:348] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:33.226714 2193 raft_consensus.cc:729] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43982f4af9314e91b1e68e0f20ad74c9, State: Initialized, Role: FOLLOWER
I20260806 18:37:33.227284 2193 consensus_queue.cc:260] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:33.228462 2193 ts_tablet_manager.cc:1428] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260806 18:37:33.242871 2304 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:37:33.200743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:36481)
0806 18:37:33.204602 (+ 3859us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:33.204678 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:33.206516 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:33.241827 (+ 35311us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":3573,"thread_start_us":123,"threads_started":1}
W20260806 18:37:33.243549 2302 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:34987 (0 consecutive failures): Network error: Failed to ping master at 127.18.125.190:34987: Client connection negotiation failed: client connection to 127.18.125.190:34987: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:37:33.244434 2304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:33.244072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:56439)
0806 18:37:33.244262 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:33.244356 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 18:37:33.245260 2304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:33.245009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:50343)
0806 18:37:33.245135 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:33.245193 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 18:37:33.245513 2302 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:37:33.327543 2305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:33.327816 2305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:33.327862 2305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:33.331969 2305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:33.332024 2305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:33.332083 2305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:37:33.337527 2305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:41465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=34285
--tserver_master_addrs=127.18.125.190:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:33.338601 2305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:33.339610 2305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:33.349318 2314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:33.349470 2305 server_base.cc:1034] running on GCE node
W20260806 18:37:33.349326 2316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:33.349332 2313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:33.350041 2305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:33.350700 2305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:33.351893 2305 hybrid_clock.cc:648] HybridClock initialized: now 1786041453351876 us; error 37 us; skew 500 ppm
Aug 06 18:37:33 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041453, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:33.354645 2305 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:37:33.355566 2305 webserver.cc:460] Webserver started at http://127.18.125.130:34285/ using document root <none> and password file <none>
I20260806 18:37:33.356102 2305 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:33.356231 2305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:33.360066 2305 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:37:33.362259 2323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:33.363502 2305 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:37:33.363608 2305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "5498db8406b44b3aa0657eeb79e81479"
format_stamp: "Formatted at 2026-08-06 18:37:31 on dist-test-slave-8w9v"
server_key: "91cb82edd7cee5b38247dfb6aba9f37b"
server_key_iv: "a0bcdc75999954ae770a70f8c029b2ed"
server_key_version: "encryptionkey@0"
I20260806 18:37:33.364245 2305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:33.373905 2305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:33.376830 2305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:33.377030 2305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:33.377641 2305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:33.378509 2305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:33.378556 2305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:33.378623 2305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:33.378664 2305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:33.388991 2305 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:41465
I20260806 18:37:33.389016 2438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:41465 every 8 connection(s)
I20260806 18:37:33.390121 2305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:37:33.394307 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:33.392527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:44877)
0806 18:37:33.393027 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:33.393145 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":131,"threads_started":1}
W20260806 18:37:33.394817 2439 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:34987 (0 consecutive failures): Network error: Failed to ping master at 127.18.125.190:34987: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
I20260806 18:37:33.395478 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:33.395271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:41819)
0806 18:37:33.395379 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:33.395426 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 18:37:33.396080 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:33.395891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:50841)
0806 18:37:33.395994 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:33.396033 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 18:37:33.396283 2439 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:37:33.396845 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 2305
I20260806 18:37:33.397279 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:36997
--local_ip_for_outbound_sockets=127.18.125.131
--tserver_master_addrs=127.18.125.190:34987
--webserver_port=46177
--webserver_interface=127.18.125.131
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:33.515722 2442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:33.516057 2442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:33.516182 2442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:33.520365 2442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:33.520447 2442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:33.520567 2442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:37:33.526285 2442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:36997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=46177
--tserver_master_addrs=127.18.125.190:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:33.527565 2442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:33.528741 2442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:33.539588 2451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:33.539656 2447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:33.539956 2448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:33.541445 2442 server_base.cc:1034] running on GCE node
I20260806 18:37:33.541887 2442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:33.542572 2442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:33.543756 2442 hybrid_clock.cc:648] HybridClock initialized: now 1786041453543736 us; error 28 us; skew 500 ppm
Aug 06 18:37:33 dist-test-slave-8w9v krb5kdc[961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041453, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:33.546813 2442 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:37:33.547832 2442 webserver.cc:460] Webserver started at http://127.18.125.131:46177/ using document root <none> and password file <none>
I20260806 18:37:33.548520 2442 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:33.548619 2442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:33.552930 2442 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:33.555395 2458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:33.556857 2442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:37:33.557013 2442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "f8920a68a635465ba34d05579edde726"
format_stamp: "Formatted at 2026-08-06 18:37:31 on dist-test-slave-8w9v"
server_key: "fc6d911e81fc6630059a3a62f8a103ff"
server_key_iv: "2a85b5dba8dcd84d9953aa5a8eb22d4f"
server_key_version: "encryptionkey@0"
I20260806 18:37:33.557629 2442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:33.574931 2442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:33.578181 2442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:33.578408 2442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:33.579083 2442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:33.579967 2442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:33.580044 2442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:33.580159 2442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:33.580216 2442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:33.589802 2442 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:36997
I20260806 18:37:33.589819 2573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:36997 every 8 connection(s)
I20260806 18:37:33.590950 2442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:37:33.595086 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 2442
I20260806 18:37:33.595175 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:33.593202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:35051)
0806 18:37:33.593711 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:33.593828 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":128,"threads_started":1}
W20260806 18:37:33.595626 2574 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:34987 (0 consecutive failures): Network error: Failed to ping master at 127.18.125.190:34987: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
I20260806 18:37:33.596295 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:33.596003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:51845)
0806 18:37:33.596153 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:33.596207 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:37:33.596899 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:33.596683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:34051)
0806 18:37:33.596788 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:33.596836 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 18:37:33.597090 2574 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:37:34.246904 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:34.246272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:48895)
0806 18:37:34.246699 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:34.246770 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":92,"threads_started":1}
I20260806 18:37:34.358389 2579 raft_consensus.cc:482] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:34.358656 2579 raft_consensus.cc:504] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:34.359932 2579 leader_election.cc:288] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8920a68a635465ba34d05579edde726 (127.18.125.131:36997), 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465)
Aug 06 18:37:34 dist-test-slave-8w9v krb5kdc[961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041453, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.130@KRBTEST.COM
Aug 06 18:37:34 dist-test-slave-8w9v krb5kdc[961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041453, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.131@KRBTEST.COM
I20260806 18:37:34.375386 2580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:34.360301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:41465 (local address 127.18.125.129:38855)
0806 18:37:34.360711 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:34.360729 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:34.360841 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:34.362519 (+ 1678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:34.362525 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:34.363154 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:34.363688 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:34.363703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:34.365173 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:34.365181 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:34.366039 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:34.366049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:34.366788 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:34.366792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:34.367050 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:34.367804 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:34.367833 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:34.369944 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:34.372376 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:34.372383 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:34.372398 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:34.372675 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:34.373058 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:34.373062 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:34.373064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:34.373214 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:34.374136 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:34.374143 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:34.374520 (+ 377us) client_negotiation.cc:769] Sending connection context
0806 18:37:34.374752 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 18:37:34.375166 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":206,"spinlock_wait_cycles":13184,"thread_start_us":87,"threads_started":1}
I20260806 18:37:34.375438 2581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:34.360405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:40603 (local address 127.18.125.131:36997)
0806 18:37:34.360891 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:34.360911 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:34.360953 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:37:34.361672 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:34.361678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:34.362144 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:34.363087 (+ 943us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:34.363878 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:34.365161 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:34.366178 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:34.366682 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:34.366868 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:34.370427 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:34.370432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:34.370443 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:34.370474 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:34.372501 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:34.373023 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:34.373031 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:34.373058 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:34.373178 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:34.373535 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:34.373539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:34.373540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:34.373993 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:34.374107 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:34.374930 (+ 823us) server_negotiation.cc:299] Negotiation successful
0806 18:37:34.375241 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":79,"threads_started":1}
I20260806 18:37:34.375443 2582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:34.360455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:38855 (local address 127.18.125.130:41465)
0806 18:37:34.360920 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:34.360940 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:34.360974 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:37:34.361710 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:34.361733 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:34.362143 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:34.362454 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:34.363872 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:34.365018 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:34.366176 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:34.366653 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:34.366842 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:34.370258 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:34.370270 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:34.370293 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:34.370362 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:34.372225 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:34.372810 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:34.372818 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:34.372829 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:34.372949 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:34.373339 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:34.373342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:34.373343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:34.373918 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:34.374070 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:34.374918 (+ 848us) server_negotiation.cc:299] Negotiation successful
0806 18:37:34.375241 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":129,"threads_started":1}
I20260806 18:37:34.375384 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:34.360231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:36997 (local address 127.18.125.129:40603)
0806 18:37:34.360452 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:34.360468 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:34.360592 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:34.362521 (+ 1929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:34.362528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:34.363156 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:34.363688 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:34.363704 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:34.365313 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:34.365317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:34.366044 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:34.366055 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:34.366820 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:34.366824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:34.367058 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:34.367804 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:34.367866 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:34.370194 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:34.372654 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:34.372659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:34.372662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:34.372896 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:34.373288 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:34.373291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:34.373294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:34.373413 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:34.374147 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:34.374150 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:34.374520 (+ 370us) client_negotiation.cc:769] Sending connection context
0806 18:37:34.374752 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 18:37:34.375223 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":303,"spinlock_wait_cycles":110592}
W20260806 18:37:34.378494 2188 leader_election.cc:341] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465): Not found: Tablet not found: b753952cc60647d4b2c82089de95ec21
W20260806 18:37:34.378712 2187 leader_election.cc:341] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8920a68a635465ba34d05579edde726 (127.18.125.131:36997): Not found: Tablet not found: b753952cc60647d4b2c82089de95ec21
I20260806 18:37:34.378814 2187 leader_election.cc:302] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43982f4af9314e91b1e68e0f20ad74c9; no voters: 5498db8406b44b3aa0657eeb79e81479, f8920a68a635465ba34d05579edde726
I20260806 18:37:34.379120 2579 raft_consensus.cc:2726] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:37:34.397399 2583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:34.396953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:41539)
0806 18:37:34.397236 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:34.397291 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":79,"threads_started":1}
I20260806 18:37:34.598181 2584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:34.597742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:54333)
0806 18:37:34.598015 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:34.598079 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260806 18:37:35.248575 2585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:35.248007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:36171)
0806 18:37:35.248394 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:35.248469 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":189,"threads_started":1}
I20260806 18:37:35.398999 2586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:35.398455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:42503)
0806 18:37:35.398806 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:35.398874 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":132,"threads_started":1}
I20260806 18:37:35.599586 2587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:35.599105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:54059)
0806 18:37:35.599400 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:35.599466 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":90,"threads_started":1}
I20260806 18:37:35.954744 2588 raft_consensus.cc:482] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:35.954908 2588 raft_consensus.cc:504] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:35.955425 2588 leader_election.cc:288] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8920a68a635465ba34d05579edde726 (127.18.125.131:36997), 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465)
W20260806 18:37:35.956465 2187 leader_election.cc:341] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8920a68a635465ba34d05579edde726 (127.18.125.131:36997): Not found: Tablet not found: b753952cc60647d4b2c82089de95ec21
W20260806 18:37:35.956636 2188 leader_election.cc:341] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465): Not found: Tablet not found: b753952cc60647d4b2c82089de95ec21
I20260806 18:37:35.956795 2188 leader_election.cc:302] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43982f4af9314e91b1e68e0f20ad74c9; no voters: 5498db8406b44b3aa0657eeb79e81479, f8920a68a635465ba34d05579edde726
I20260806 18:37:35.957008 2588 raft_consensus.cc:2726] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:37:36.249987 2589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:36.249479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:39057)
0806 18:37:36.249810 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:36.249879 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":88,"threads_started":1}
I20260806 18:37:36.400454 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:36.399902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:37095)
0806 18:37:36.400282 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:36.400348 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":102,"threads_started":1}
I20260806 18:37:36.601397 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:36.600843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:35161)
0806 18:37:36.601205 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:36.601285 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":106,"threads_started":1}
I20260806 18:37:37.251987 2592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:37.251168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:35351)
0806 18:37:37.251697 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:37.251818 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":99,"threads_started":1}
I20260806 18:37:37.401899 2593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:37.401457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:42059)
0806 18:37:37.401735 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:37.401792 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260806 18:37:37.602751 2594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:37.602286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:45911)
0806 18:37:37.602589 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:37.602649 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":90,"threads_started":1}
I20260806 18:37:38.254398 2595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:38.253770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:48995)
0806 18:37:38.254189 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:38.254266 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":91,"threads_started":1}
I20260806 18:37:38.394474 2597 raft_consensus.cc:482] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:38.394610 2597 raft_consensus.cc:504] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:38.395125 2597 leader_election.cc:288] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8920a68a635465ba34d05579edde726 (127.18.125.131:36997), 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465)
W20260806 18:37:38.396067 2187 leader_election.cc:341] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8920a68a635465ba34d05579edde726 (127.18.125.131:36997): Not found: Tablet not found: b753952cc60647d4b2c82089de95ec21
W20260806 18:37:38.396179 2188 leader_election.cc:341] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465): Not found: Tablet not found: b753952cc60647d4b2c82089de95ec21
I20260806 18:37:38.396239 2187 leader_election.cc:302] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43982f4af9314e91b1e68e0f20ad74c9; no voters: 5498db8406b44b3aa0657eeb79e81479, f8920a68a635465ba34d05579edde726
I20260806 18:37:38.396414 2597 raft_consensus.cc:2726] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:37:38.403396 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:38.402897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:50093)
0806 18:37:38.403217 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:38.403284 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":116,"threads_started":1}
I20260806 18:37:38.604085 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:38.603592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:39293)
0806 18:37:38.603891 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:38.603950 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":102,"threads_started":1}
I20260806 18:37:39.256217 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:39.255579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:46651)
0806 18:37:39.255941 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:39.256037 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":111,"threads_started":1}
I20260806 18:37:39.404845 2601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:39.404385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:54277)
0806 18:37:39.404668 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:39.404728 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":81,"threads_started":1}
I20260806 18:37:39.605465 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:39.604926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:59667)
0806 18:37:39.605214 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:39.605300 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":78,"threads_started":1}
I20260806 18:37:40.257666 2603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:40.257171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:43157)
0806 18:37:40.257497 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:40.257563 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":85,"threads_started":1}
I20260806 18:37:40.406493 2604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:40.405930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:40479)
0806 18:37:40.406292 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:40.406359 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":83,"threads_started":1}
I20260806 18:37:40.606978 2605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:40.606475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:45989)
0806 18:37:40.606809 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:40.606874 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":81,"threads_started":1}
I20260806 18:37:41.259140 2606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:41.258619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:58915)
0806 18:37:41.258954 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:41.259036 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":104,"threads_started":1}
I20260806 18:37:41.407752 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:41.407315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:46039)
0806 18:37:41.407583 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:41.407648 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":78,"threads_started":1}
I20260806 18:37:41.608309 2608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:41.607804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:48945)
0806 18:37:41.608147 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:41.608207 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":113,"threads_started":1}
I20260806 18:37:42.260855 2609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:42.260219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:56427)
0806 18:37:42.260643 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:42.260733 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":114,"threads_started":1}
I20260806 18:37:42.409230 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:42.408717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:44281)
0806 18:37:42.409053 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:42.409119 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":108,"threads_started":1}
I20260806 18:37:42.609633 2611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:42.609162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:54965)
0806 18:37:42.609462 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:42.609528 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":82,"threads_started":1}
I20260806 18:37:43.156528 2612 raft_consensus.cc:482] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:43.156695 2612 raft_consensus.cc:504] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:43.157186 2612 leader_election.cc:288] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8920a68a635465ba34d05579edde726 (127.18.125.131:36997), 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465)
W20260806 18:37:43.158041 2187 leader_election.cc:341] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8920a68a635465ba34d05579edde726 (127.18.125.131:36997): Not found: Tablet not found: b753952cc60647d4b2c82089de95ec21
W20260806 18:37:43.158138 2188 leader_election.cc:341] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465): Not found: Tablet not found: b753952cc60647d4b2c82089de95ec21
I20260806 18:37:43.158260 2188 leader_election.cc:302] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43982f4af9314e91b1e68e0f20ad74c9; no voters: 5498db8406b44b3aa0657eeb79e81479, f8920a68a635465ba34d05579edde726
I20260806 18:37:43.158433 2612 raft_consensus.cc:2726] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:37:43.262406 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:43.261931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:36927)
0806 18:37:43.262224 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:43.262281 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":112,"threads_started":1}
I20260806 18:37:43.410477 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:43.410035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:52451)
0806 18:37:43.410314 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:43.410377 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":75,"threads_started":1}
I20260806 18:37:43.611030 2615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:43.610511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:45097)
0806 18:37:43.610854 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:43.610923 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":104,"threads_started":1}
I20260806 18:37:44.263768 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:44.263274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:49343)
0806 18:37:44.263596 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:44.263664 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":82,"threads_started":1}
I20260806 18:37:44.412228 2617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:44.411450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:42873)
0806 18:37:44.411790 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:44.411867 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":123,"threads_started":1}
I20260806 18:37:44.612389 2618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:44.611860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:34755)
0806 18:37:44.612218 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:44.612285 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
I20260806 18:37:45.265313 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:45.264830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:49135)
0806 18:37:45.265131 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:45.265198 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":92,"threads_started":1}
I20260806 18:37:45.413667 2620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:45.413184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:53945)
0806 18:37:45.413492 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:45.413561 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":83,"threads_started":1}
I20260806 18:37:45.613830 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:45.613304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:33653)
0806 18:37:45.613620 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:45.613708 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":102,"threads_started":1}
I20260806 18:37:46.266791 2622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:46.266276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:38215)
0806 18:37:46.266616 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:46.266685 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":108,"threads_started":1}
I20260806 18:37:46.415395 2623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:46.414818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:50329)
0806 18:37:46.415200 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:46.415283 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":98,"threads_started":1}
I20260806 18:37:46.615417 2624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:46.614887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:60627)
0806 18:37:46.615215 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:46.615283 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":97,"threads_started":1}
I20260806 18:37:46.841274 2625 raft_consensus.cc:482] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:46.841419 2625 raft_consensus.cc:504] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8920a68a635465ba34d05579edde726" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 36997 } } peers { permanent_uuid: "5498db8406b44b3aa0657eeb79e81479" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 41465 } } peers { permanent_uuid: "43982f4af9314e91b1e68e0f20ad74c9" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 43067 } }
I20260806 18:37:46.841810 2625 leader_election.cc:288] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8920a68a635465ba34d05579edde726 (127.18.125.131:36997), 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465)
W20260806 18:37:46.842626 2188 leader_election.cc:341] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5498db8406b44b3aa0657eeb79e81479 (127.18.125.130:41465): Not found: Tablet not found: b753952cc60647d4b2c82089de95ec21
W20260806 18:37:46.842757 2187 leader_election.cc:341] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8920a68a635465ba34d05579edde726 (127.18.125.131:36997): Not found: Tablet not found: b753952cc60647d4b2c82089de95ec21
I20260806 18:37:46.842860 2187 leader_election.cc:302] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43982f4af9314e91b1e68e0f20ad74c9; no voters: 5498db8406b44b3aa0657eeb79e81479, f8920a68a635465ba34d05579edde726
I20260806 18:37:46.843034 2625 raft_consensus.cc:2726] T b753952cc60647d4b2c82089de95ec21 P 43982f4af9314e91b1e68e0f20ad74c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:37:47.268460 2626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:47.267814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:52599)
0806 18:37:47.268236 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:47.268317 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":161,"threads_started":1}
I20260806 18:37:47.416870 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:47.416383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:60197)
0806 18:37:47.416694 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:47.416763 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":88,"threads_started":1}
I20260806 18:37:47.616909 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:47.616446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:38225)
0806 18:37:47.616732 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:47.616796 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":83,"threads_started":1}
I20260806 18:37:48.270051 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:48.269645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.129:48621)
0806 18:37:48.269893 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:48.269952 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":77,"threads_started":1}
I20260806 18:37:48.418435 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:48.417938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.130:42985)
0806 18:37:48.418230 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:48.418333 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
I20260806 18:37:48.600306 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 2135
I20260806 18:37:48.610409 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 2305
I20260806 18:37:48.618407 2631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:48.617961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:34987 (local address 127.18.125.131:49613)
0806 18:37:48.618247 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:48.618307 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":95,"threads_started":1}
I20260806 18:37:48.618450 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 2442
2026-08-06T18:37:48Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19452 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:37:48 dist-test-slave-8w9v krb5kdc[2634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:37:48 dist-test-slave-8w9v krb5kdc[2634](info): set up 2 sockets
Aug 06 18:37:48 dist-test-slave-8w9v krb5kdc[2634](info): commencing operation
krb5kdc: starting...
W20260806 18:37:49.706509 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:37:49 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:37:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:37:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:37:49.844316 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:39117
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:39117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:49.960245 2647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:49.960644 2647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:49.960727 2647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:49.965025 2647 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:37:49.965135 2647 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:37:49.965159 2647 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:37:49.965183 2647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:37:49.965227 2647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:49.965294 2647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:49.965348 2647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:37:49.965386 2647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:37:49.971165 2647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:39117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:39117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:49.972478 2647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:49.973614 2647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:49.983390 2652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:49.983477 2647 server_base.cc:1034] running on GCE node
W20260806 18:37:49.983404 2653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:49.983423 2655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:49.984520 2647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:49.985714 2647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:49.986896 2647 hybrid_clock.cc:648] HybridClock initialized: now 1786041469986867 us; error 50 us; skew 500 ppm
Aug 06 18:37:49 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041469, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:49.990080 2647 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:37:49.991119 2647 webserver.cc:460] Webserver started at http://127.18.125.190:32887/ using document root <none> and password file <none>
I20260806 18:37:49.991726 2647 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:49.991794 2647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:49.992031 2647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:49.994342 2647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "106f0a802ca94b2db35c7adf7b1d5bcb"
format_stamp: "Formatted at 2026-08-06 18:37:49 on dist-test-slave-8w9v"
server_key: "39fed4bc5f09b54d582f754f2b51e5fc"
server_key_iv: "b20a445f813a70afcb2d9f742b2c2fd5"
server_key_version: "encryptionkey@0"
I20260806 18:37:49.994925 2647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "106f0a802ca94b2db35c7adf7b1d5bcb"
format_stamp: "Formatted at 2026-08-06 18:37:49 on dist-test-slave-8w9v"
server_key: "39fed4bc5f09b54d582f754f2b51e5fc"
server_key_iv: "b20a445f813a70afcb2d9f742b2c2fd5"
server_key_version: "encryptionkey@0"
I20260806 18:37:49.998926 2647 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:37:50.001593 2662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:50.002884 2647 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:37:50.002995 2647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "106f0a802ca94b2db35c7adf7b1d5bcb"
format_stamp: "Formatted at 2026-08-06 18:37:49 on dist-test-slave-8w9v"
server_key: "39fed4bc5f09b54d582f754f2b51e5fc"
server_key_iv: "b20a445f813a70afcb2d9f742b2c2fd5"
server_key_version: "encryptionkey@0"
I20260806 18:37:50.003088 2647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:50.018064 2647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:50.021965 2647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:50.022173 2647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:50.029862 2647 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:39117
I20260806 18:37:50.030154 2714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:39117 every 8 connection(s)
I20260806 18:37:50.030978 2647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:37:50.031728 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 2647
I20260806 18:37:50.031867 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:37:50.032239 18934 external_mini_cluster.cc:1428] Setting key 13d4fe9675239f6772055f65017bcfd6
I20260806 18:37:50.034368 2715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:50.041429 2715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Bootstrap starting.
Aug 06 18:37:50 dist-test-slave-8w9v krb5kdc[2634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:50.044240 2715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:50.045099 2715 log.cc:824] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:50.047189 2715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: No bootstrap required, opened a new log
I20260806 18:37:50.048574 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.033810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59720 (local address 127.18.125.190:39117)
0806 18:37:50.034276 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:50.034296 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:50.034340 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:37:50.035212 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:50.035235 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:50.035646 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:50.036034 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:50.036854 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.038399 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.039358 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.040060 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.040267 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.043136 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:50.043143 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:50.043160 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:50.043198 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:50.045365 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:50.045882 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:50.045894 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:50.045900 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:50.046028 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:50.046343 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:50.046348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:50.046350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:50.046803 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:50.047037 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:50.047359 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 18:37:50.047703 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":130,"threads_started":1}
I20260806 18:37:50.051379 2715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:37:50.051645 2715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:50.051717 2715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106f0a802ca94b2db35c7adf7b1d5bcb, State: Initialized, Role: FOLLOWER
I20260806 18:37:50.052223 2715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:37:50.052392 2715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:37:50.052457 2715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:37:50.052554 2715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:50.053586 2715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:37:50.053972 2715 leader_election.cc:302] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 106f0a802ca94b2db35c7adf7b1d5bcb; no voters:
I20260806 18:37:50.054315 2715 leader_election.cc:288] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:37:50.054478 2720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:50.054770 2720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 1 LEADER]: Becoming Leader. State: Replica: 106f0a802ca94b2db35c7adf7b1d5bcb, State: Running, Role: LEADER
I20260806 18:37:50.055094 2720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:37:50.055596 2715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:37:50.057015 2721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } } }
I20260806 18:37:50.057039 2722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 106f0a802ca94b2db35c7adf7b1d5bcb. Latest consensus state: current_term: 1 leader_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } } }
I20260806 18:37:50.057183 2721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: This master's current role is: LEADER
I20260806 18:37:50.057183 2722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: This master's current role is: LEADER
W20260806 18:37:50.060561 2736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:37:50.060652 2736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:37:50.060722 2732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:37:50.061414 2732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:37:50.067405 2732 catalog_manager.cc:1347] Generated new cluster ID: 62e710d393ea4ea29b35839b8c4edd6a
I20260806 18:37:50.067498 2732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:37:50.085263 2732 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:37:50.086378 2732 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:37:50.096787 2732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Generated new TSK 0
I20260806 18:37:50.097617 2732 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:37:50.131933 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:39117
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:50.257776 2742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:50.258095 2742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:50.258180 2742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:50.262699 2742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:50.262785 2742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:50.262884 2742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:37:50.268761 2742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:50.270164 2742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:50.271615 2742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:50.282313 2747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:50.282385 2748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:50.282313 2750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:50.282810 2742 server_base.cc:1034] running on GCE node
I20260806 18:37:50.283283 2742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:50.284193 2742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:50.285513 2742 hybrid_clock.cc:648] HybridClock initialized: now 1786041470285471 us; error 36 us; skew 500 ppm
Aug 06 18:37:50 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:50.289717 2742 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:37:50.290848 2742 webserver.cc:460] Webserver started at http://127.18.125.129:45079/ using document root <none> and password file <none>
I20260806 18:37:50.291494 2742 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:50.291586 2742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:50.291837 2742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:50.294251 2742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "06d3d04d41944752bae1c881c3a2d526"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "31952d7532bbc63aa2c3363ea1071c5d"
server_key_iv: "3eb62e1ce5dd0682f1df32c8f09c21c5"
server_key_version: "encryptionkey@0"
I20260806 18:37:50.294878 2742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "06d3d04d41944752bae1c881c3a2d526"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "31952d7532bbc63aa2c3363ea1071c5d"
server_key_iv: "3eb62e1ce5dd0682f1df32c8f09c21c5"
server_key_version: "encryptionkey@0"
I20260806 18:37:50.299003 2742 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:37:50.302047 2757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:50.303550 2742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:37:50.303684 2742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "06d3d04d41944752bae1c881c3a2d526"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "31952d7532bbc63aa2c3363ea1071c5d"
server_key_iv: "3eb62e1ce5dd0682f1df32c8f09c21c5"
server_key_version: "encryptionkey@0"
I20260806 18:37:50.303803 2742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:50.318060 2742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:50.321310 2742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:50.321516 2742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:50.322161 2742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:50.323004 2742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:50.323051 2742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:50.323117 2742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:50.323148 2742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:50.331928 2742 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:34633
I20260806 18:37:50.331947 2870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:34633 every 8 connection(s)
I20260806 18:37:50.332834 2742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:37:50.340236 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 2742
I20260806 18:37:50.340370 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:37:50.340749 18934 external_mini_cluster.cc:1428] Setting key 1bbf075f1891ec1088e91c148b2d3677
Aug 06 18:37:50 dist-test-slave-8w9v krb5kdc[2634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:50.347404 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.334663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:35737 (local address 127.18.125.190:39117)
0806 18:37:50.334803 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:50.334806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:50.335555 (+ 749us) server_negotiation.cc:407] Connection header received
0806 18:37:50.337010 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:50.337014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:50.337076 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:50.337179 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:50.338342 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.338885 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.340074 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.340454 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.340569 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.343403 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:50.343407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:50.343409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:50.343434 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:50.344941 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:50.345594 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:50.345601 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:50.345605 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:50.345702 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:50.346045 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:50.346051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:50.346056 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:50.346321 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:50.346466 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:50.346964 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 18:37:50.347136 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:37:50.348178 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.334968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:35737)
0806 18:37:50.335411 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:50.335453 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:50.336787 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:50.337353 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:50.337360 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:50.337720 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:50.338192 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.338205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.339023 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.339026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.339922 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.339939 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.340633 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.340637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.340857 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.341600 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:50.341621 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:50.343213 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:50.345067 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:50.345077 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:50.345094 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:50.345481 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:50.345802 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:50.345804 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:50.345805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:50.345952 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:50.346466 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:50.346471 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:50.346710 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 18:37:50.346871 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 18:37:50.347210 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":116,"threads_started":1}
I20260806 18:37:50.349502 2871 heartbeater.cc:347] Connected to a master server at 127.18.125.190:39117
I20260806 18:37:50.349782 2871 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:50.350451 2871 heartbeater.cc:511] Master 127.18.125.190:39117 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:37:50.352191 2679 ts_manager.cc:194] Registered new tserver with Master: 06d3d04d41944752bae1c881c3a2d526 (127.18.125.129:34633)
I20260806 18:37:50.353359 2679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:35737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:37:50.371138 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:39117
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:50.487033 2876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:50.487313 2876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:50.487360 2876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:50.491468 2876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:50.491535 2876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:50.491598 2876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:37:50.497596 2876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:50.498708 2876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:50.499715 2876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:50.509882 2882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:50.509984 2881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:50.509915 2884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:50.510427 2876 server_base.cc:1034] running on GCE node
I20260806 18:37:50.510823 2876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:50.511492 2876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:50.512676 2876 hybrid_clock.cc:648] HybridClock initialized: now 1786041470512660 us; error 27 us; skew 500 ppm
Aug 06 18:37:50 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:50.515589 2876 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:37:50.516542 2876 webserver.cc:460] Webserver started at http://127.18.125.130:32865/ using document root <none> and password file <none>
I20260806 18:37:50.517138 2876 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:50.517231 2876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:50.517468 2876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:50.519569 2876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "6df4b687354d41049a4310cc028718cd"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "5fec30685d9812b520f33558464f2ac7"
server_key_iv: "84fac2e17495733576667738a4352fc0"
server_key_version: "encryptionkey@0"
I20260806 18:37:50.520200 2876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "6df4b687354d41049a4310cc028718cd"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "5fec30685d9812b520f33558464f2ac7"
server_key_iv: "84fac2e17495733576667738a4352fc0"
server_key_version: "encryptionkey@0"
I20260806 18:37:50.524111 2876 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 18:37:50.526809 2891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:50.528091 2876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:50.528286 2876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "6df4b687354d41049a4310cc028718cd"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "5fec30685d9812b520f33558464f2ac7"
server_key_iv: "84fac2e17495733576667738a4352fc0"
server_key_version: "encryptionkey@0"
I20260806 18:37:50.528422 2876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:50.538661 2876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:50.541705 2876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:50.541934 2876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:50.542577 2876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:50.543485 2876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:50.543557 2876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:50.543632 2876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:50.543673 2876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:50.553409 2876 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:37439
I20260806 18:37:50.553428 3006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:37439 every 8 connection(s)
I20260806 18:37:50.554360 2876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:37:50.558262 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 2876
I20260806 18:37:50.558409 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:37:50.558703 18934 external_mini_cluster.cc:1428] Setting key 75c61a4277b2389f0ad91f726c6500ed
Aug 06 18:37:50 dist-test-slave-8w9v krb5kdc[2634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:37:50.569682 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.556325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:34691 (local address 127.18.125.190:39117)
0806 18:37:50.556496 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:50.556501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:50.557472 (+ 971us) server_negotiation.cc:407] Connection header received
0806 18:37:50.559345 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:50.559348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:50.559406 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:50.559490 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:50.561015 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.561542 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.562875 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.563115 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.563223 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.565864 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:50.565868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:50.565870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:50.565909 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:50.567407 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:50.567959 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:50.567962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:50.567969 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:50.568045 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:50.568474 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:50.568477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:50.568479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:50.568679 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:50.568783 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:50.569429 (+ 646us) server_negotiation.cc:299] Negotiation successful
0806 18:37:50.569551 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:37:50.570676 3009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.556814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:34691)
0806 18:37:50.557313 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:50.557357 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:50.559122 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:50.559670 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:50.559677 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:50.560055 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:50.560827 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.560852 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.561682 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.561689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.562739 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.562750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.563281 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.563285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.563462 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.564102 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:50.564163 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:50.565680 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:50.567537 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:50.567543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:50.567561 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:50.567836 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:50.568163 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:50.568166 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:50.568168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:50.568364 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:50.568797 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:50.568806 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:50.569078 (+ 272us) client_negotiation.cc:769] Sending connection context
0806 18:37:50.569343 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 18:37:50.569663 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
I20260806 18:37:50.572093 3007 heartbeater.cc:347] Connected to a master server at 127.18.125.190:39117
I20260806 18:37:50.572425 3007 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:50.572986 3007 heartbeater.cc:511] Master 127.18.125.190:39117 requested a full tablet report, sending...
I20260806 18:37:50.574451 2679 ts_manager.cc:194] Registered new tserver with Master: 6df4b687354d41049a4310cc028718cd (127.18.125.130:37439)
I20260806 18:37:50.575147 2679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:34691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:37:50.589751 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:39117
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:37:50.708613 3012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:37:50.708909 3012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:37:50.708997 3012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:37:50.713338 3012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:37:50.713429 3012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:37:50.713502 3012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:37:50.719668 3012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:37:50.721086 3012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:37:50.722277 3012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:37:50.732977 3018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:50.732977 3017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:37:50.732980 3020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:37:50.733217 3012 server_base.cc:1034] running on GCE node
I20260806 18:37:50.733696 3012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:37:50.734416 3012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:37:50.735627 3012 hybrid_clock.cc:648] HybridClock initialized: now 1786041470735609 us; error 33 us; skew 500 ppm
Aug 06 18:37:50 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:37:50.738686 3012 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:37:50.739869 3012 webserver.cc:460] Webserver started at http://127.18.125.131:37163/ using document root <none> and password file <none>
I20260806 18:37:50.740566 3012 fs_manager.cc:359] Metadata directory not provided
I20260806 18:37:50.740656 3012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:37:50.740903 3012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:37:50.743043 3012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "9879ac85d2854485bee53d9a65de24e4"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "60b5d89fec6a3ca050c2e05fd3954fef"
server_key_iv: "17f1e55c23cc626e186d74a1aa3c5948"
server_key_version: "encryptionkey@0"
I20260806 18:37:50.743624 3012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "9879ac85d2854485bee53d9a65de24e4"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "60b5d89fec6a3ca050c2e05fd3954fef"
server_key_iv: "17f1e55c23cc626e186d74a1aa3c5948"
server_key_version: "encryptionkey@0"
I20260806 18:37:50.747462 3012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 18:37:50.750051 3027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:50.751353 3012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:37:50.751485 3012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "9879ac85d2854485bee53d9a65de24e4"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "60b5d89fec6a3ca050c2e05fd3954fef"
server_key_iv: "17f1e55c23cc626e186d74a1aa3c5948"
server_key_version: "encryptionkey@0"
I20260806 18:37:50.751597 3012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:37:50.766743 3012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:37:50.770125 3012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:37:50.770330 3012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:37:50.770943 3012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:37:50.771773 3012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:37:50.771842 3012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:50.771936 3012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:37:50.771986 3012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:37:50.781507 3012 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:33339
I20260806 18:37:50.781526 3144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:33339 every 8 connection(s)
I20260806 18:37:50.782449 3012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:37:50.787559 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 3012
I20260806 18:37:50.787727 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:37:50.788030 18934 external_mini_cluster.cc:1428] Setting key 4a9ff2b5c640168a7ae8ca75f9bf65c5
Aug 06 18:37:50 dist-test-slave-8w9v krb5kdc[2634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041470, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:37:50.796923 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.784352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:49311 (local address 127.18.125.190:39117)
0806 18:37:50.784497 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:50.784501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:50.785386 (+ 885us) server_negotiation.cc:407] Connection header received
0806 18:37:50.786777 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:50.786781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:50.786844 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:50.786949 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:50.788644 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.789396 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.790542 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.790775 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.790886 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.793390 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:50.793394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:50.793397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:50.793431 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:50.794651 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:50.795189 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:50.795193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:50.795194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:50.795310 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:50.795671 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:50.795674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:50.795675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:50.795873 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:50.795988 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:50.796613 (+ 625us) server_negotiation.cc:299] Negotiation successful
0806 18:37:50.796751 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:37:50.797744 3147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.784773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:49311)
0806 18:37:50.785234 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:50.785279 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:50.786560 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:50.787128 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:50.787135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:50.787742 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:37:50.788492 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.788511 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.789510 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.789513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.790416 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.790427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.790933 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.790937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.791116 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.791663 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:37:50.791680 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:37:50.793218 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:37:50.794781 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:50.794787 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:50.794803 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:50.795080 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:50.795406 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:37:50.795411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:37:50.795413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:37:50.795566 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:37:50.796001 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:37:50.796009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:37:50.796365 (+ 356us) client_negotiation.cc:769] Sending connection context
0806 18:37:50.796553 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 18:37:50.796911 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":117,"threads_started":1}
I20260806 18:37:50.798880 3145 heartbeater.cc:347] Connected to a master server at 127.18.125.190:39117
I20260806 18:37:50.799144 3145 heartbeater.cc:464] Registering TS with master...
I20260806 18:37:50.799677 3145 heartbeater.cc:511] Master 127.18.125.190:39117 requested a full tablet report, sending...
I20260806 18:37:50.800823 2679 ts_manager.cc:194] Registered new tserver with Master: 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339)
I20260806 18:37:50.801542 2679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:49311
I20260806 18:37:50.802124 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:37:50.812085 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.804224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.18.125.190:39117)
0806 18:37:50.804399 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:50.804403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:50.804510 (+ 107us) server_negotiation.cc:407] Connection header received
0806 18:37:50.804643 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:50.804645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:50.804683 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:50.804750 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:37:50.805573 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.806152 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.807087 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.807321 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.807432 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.808987 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:37:50.808990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:37:50.808992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:37:50.809018 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:37:50.810327 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:50.810758 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:50.810762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:50.810764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:50.810849 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:37:50.811096 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:37:50.811098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:37:50.811100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:37:50.811352 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:37:50.811467 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:50.811715 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 18:37:50.811838 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 18:37:50.816573 2678 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:37:50.821092 2678 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:37:50.838475 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.832313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49834 (local address 127.18.125.130:37439)
0806 18:37:50.832711 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:50.832725 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:50.833428 (+ 703us) server_negotiation.cc:407] Connection header received
0806 18:37:50.833585 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:50.833595 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:50.833787 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:50.833980 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:50.834493 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.835796 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.837193 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.837750 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.837891 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.838037 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:50.838173 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 18:37:50.838278 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":113,"threads_started":1}
I20260806 18:37:50.838462 3156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.830846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:33339 (local address 127.0.0.1:52482)
0806 18:37:50.831604 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:50.831660 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:50.831798 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:50.833320 (+ 1522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:50.833326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:50.833356 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:50.833521 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.833530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.835345 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.835348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.836814 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.836825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.837938 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.837942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.838043 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.838065 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:37:50.838119 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 18:37:50.838277 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":1,"thread_start_us":203,"threads_started":1}
I20260806 18:37:50.838631 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.830863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:34633 (local address 127.0.0.1:43596)
0806 18:37:50.832233 (+ 1370us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:50.832249 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:50.832352 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:50.833656 (+ 1304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:50.833661 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:50.833682 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:50.833756 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.833760 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.835731 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.835735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.836776 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.836786 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.837938 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.837942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.838283 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.838304 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:37:50.838378 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:37:50.838434 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1288,"mutex_wait_us":234,"thread_start_us":147,"threads_started":1}
I20260806 18:37:50.839171 3159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.831573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52482 (local address 127.18.125.131:33339)
0806 18:37:50.832473 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:50.832500 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:50.832639 (+ 139us) server_negotiation.cc:407] Connection header received
0806 18:37:50.832745 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:50.832751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:50.833036 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:50.833340 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:50.833863 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.835193 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.836961 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.837788 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.838706 (+ 918us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.838879 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:50.838936 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 18:37:50.839029 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":184,"threads_started":1}
I20260806 18:37:50.839198 3158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.831036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43596 (local address 127.18.125.129:34633)
0806 18:37:50.832808 (+ 1772us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:50.832824 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:50.832847 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:37:50.832937 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:50.832945 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:50.833460 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:50.833676 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:50.834218 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.835556 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.836912 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.837782 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.838006 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.838143 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:50.838949 (+ 806us) server_negotiation.cc:299] Negotiation successful
0806 18:37:50.839045 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1661,"thread_start_us":108,"threads_started":1}
I20260806 18:37:50.839407 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:50.831677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:37439 (local address 127.0.0.1:49834)
0806 18:37:50.832887 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:50.832900 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:50.833209 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:50.833955 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:50.833957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:50.834060 (+ 103us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:50.834317 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.834331 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.835943 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.835946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.837076 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:50.837084 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:50.837874 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:50.837878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:50.837986 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:50.838024 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:37:50.838140 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 18:37:50.839250 (+ 1110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1142,"mutex_wait_us":976,"thread_start_us":100,"threads_started":1}
I20260806 18:37:50.842027 2805 tablet_service.cc:1467] Processing CreateTablet for tablet b858322b8e014d4fb35b5f10a06ff11b (DEFAULT_TABLE table=test-table [id=be5e53d9227843b69f02b621530842cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:50.842126 3077 tablet_service.cc:1467] Processing CreateTablet for tablet b858322b8e014d4fb35b5f10a06ff11b (DEFAULT_TABLE table=test-table [id=be5e53d9227843b69f02b621530842cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:50.842127 2940 tablet_service.cc:1467] Processing CreateTablet for tablet b858322b8e014d4fb35b5f10a06ff11b (DEFAULT_TABLE table=test-table [id=be5e53d9227843b69f02b621530842cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:37:50.843255 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b858322b8e014d4fb35b5f10a06ff11b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:50.843267 2940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b858322b8e014d4fb35b5f10a06ff11b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:50.843268 3077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b858322b8e014d4fb35b5f10a06ff11b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:37:50.849713 3162 tablet_bootstrap.cc:492] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Bootstrap starting.
I20260806 18:37:50.851156 3164 tablet_bootstrap.cc:492] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: Bootstrap starting.
I20260806 18:37:50.851173 3163 tablet_bootstrap.cc:492] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: Bootstrap starting.
I20260806 18:37:50.852089 3162 tablet_bootstrap.cc:654] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:50.853396 3163 tablet_bootstrap.cc:654] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:50.853397 3162 log.cc:824] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:50.853928 3164 tablet_bootstrap.cc:654] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: Neither blocks nor log segments found. Creating new log.
I20260806 18:37:50.854753 3164 log.cc:824] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:50.854763 3163 log.cc:824] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: Log is configured to *not* fsync() on all Append() calls
I20260806 18:37:50.855499 3162 tablet_bootstrap.cc:492] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: No bootstrap required, opened a new log
I20260806 18:37:50.855763 3162 ts_tablet_manager.cc:1397] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:37:50.857379 3164 tablet_bootstrap.cc:492] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: No bootstrap required, opened a new log
I20260806 18:37:50.857441 3163 tablet_bootstrap.cc:492] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: No bootstrap required, opened a new log
I20260806 18:37:50.857607 3164 ts_tablet_manager.cc:1397] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 18:37:50.857607 3163 ts_tablet_manager.cc:1397] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260806 18:37:50.859805 3162 raft_consensus.cc:348] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } }
I20260806 18:37:50.860131 3162 raft_consensus.cc:374] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:50.860234 3162 raft_consensus.cc:729] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d3d04d41944752bae1c881c3a2d526, State: Initialized, Role: FOLLOWER
I20260806 18:37:50.861040 3162 consensus_queue.cc:260] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } }
I20260806 18:37:50.861768 2871 heartbeater.cc:503] Master 127.18.125.190:39117 was elected leader, sending a full tablet report...
I20260806 18:37:50.861625 3164 raft_consensus.cc:348] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } }
I20260806 18:37:50.861625 3163 raft_consensus.cc:348] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } }
I20260806 18:37:50.861908 3164 raft_consensus.cc:374] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:50.861908 3163 raft_consensus.cc:374] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:37:50.861971 3164 raft_consensus.cc:729] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df4b687354d41049a4310cc028718cd, State: Initialized, Role: FOLLOWER
I20260806 18:37:50.861979 3163 raft_consensus.cc:729] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9879ac85d2854485bee53d9a65de24e4, State: Initialized, Role: FOLLOWER
I20260806 18:37:50.862399 3163 consensus_queue.cc:260] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } }
I20260806 18:37:50.862396 3164 consensus_queue.cc:260] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } }
I20260806 18:37:50.863353 3145 heartbeater.cc:503] Master 127.18.125.190:39117 was elected leader, sending a full tablet report...
I20260806 18:37:50.863353 3007 heartbeater.cc:503] Master 127.18.125.190:39117 was elected leader, sending a full tablet report...
I20260806 18:37:50.863435 3163 ts_tablet_manager.cc:1428] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:37:50.863435 3164 ts_tablet_manager.cc:1428] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:37:50.865612 3162 ts_tablet_manager.cc:1428] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
W20260806 18:37:51.033440 3146 tablet.cc:2369] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:37:51.055625 3008 tablet.cc:2369] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:37:51.084424 2872 tablet.cc:2369] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:37:51.157596 3170 raft_consensus.cc:482] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:37:51.157871 3170 raft_consensus.cc:504] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } }
I20260806 18:37:51.159144 3170 leader_election.cc:288] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06d3d04d41944752bae1c881c3a2d526 (127.18.125.129:34633), 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339)
I20260806 18:37:51.163353 3159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:51.159729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:37001 (local address 127.18.125.131:33339)
0806 18:37:51.159864 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:51.159869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:51.159952 (+ 83us) server_negotiation.cc:407] Connection header received
0806 18:37:51.160199 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:51.160203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:51.160258 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:51.160350 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:51.160671 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:51.161179 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:51.162323 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:51.162747 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:51.162890 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:51.162965 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:51.163133 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 18:37:51.163218 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:37:51.163352 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:51.159535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:33339 (local address 127.18.125.130:37001)
0806 18:37:51.159845 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:51.159857 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:51.159980 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:51.160379 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:51.160381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:51.160396 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:51.160553 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:51.160560 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:51.161299 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:51.161303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:51.162186 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:51.162196 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:51.162876 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:51.162880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:51.162980 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:51.163012 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 18:37:51.163082 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:37:51.163167 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":13,"thread_start_us":104,"threads_started":1}
I20260806 18:37:51.163446 3158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:51.159533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:58257 (local address 127.18.125.129:34633)
0806 18:37:51.159696 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:51.159701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:51.159786 (+ 85us) server_negotiation.cc:407] Connection header received
0806 18:37:51.160041 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:51.160048 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:51.160109 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:51.160193 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:37:51.160548 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:51.161110 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:51.162378 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:51.162777 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:51.162920 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:51.162988 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:51.163203 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 18:37:51.163287 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:37:51.163372 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:51.159393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:34633 (local address 127.18.125.130:58257)
0806 18:37:51.159682 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:51.159698 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:37:51.159892 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:37:51.160225 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:37:51.160228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:37:51.160254 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:37:51.160416 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:51.160424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:51.161236 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:51.161243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:51.162193 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:37:51.162204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:37:51.162934 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:37:51.162937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:37:51.163048 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:51.163086 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:37:51.163168 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:37:51.163233 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":95,"thread_start_us":81,"threads_started":1}
I20260806 18:37:51.164567 2825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858322b8e014d4fb35b5f10a06ff11b" candidate_uuid: "6df4b687354d41049a4310cc028718cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d3d04d41944752bae1c881c3a2d526" is_pre_election: true
I20260806 18:37:51.164563 3099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858322b8e014d4fb35b5f10a06ff11b" candidate_uuid: "6df4b687354d41049a4310cc028718cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9879ac85d2854485bee53d9a65de24e4" is_pre_election: true
I20260806 18:37:51.164877 3099 raft_consensus.cc:2454] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6df4b687354d41049a4310cc028718cd in term 0.
I20260806 18:37:51.164884 2825 raft_consensus.cc:2454] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6df4b687354d41049a4310cc028718cd in term 0.
I20260806 18:37:51.165395 2893 leader_election.cc:302] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 06d3d04d41944752bae1c881c3a2d526, 6df4b687354d41049a4310cc028718cd, 9879ac85d2854485bee53d9a65de24e4; no voters:
I20260806 18:37:51.165694 3170 raft_consensus.cc:2781] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:37:51.165788 3170 raft_consensus.cc:482] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:37:51.165831 3170 raft_consensus.cc:3037] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:51.166895 3170 raft_consensus.cc:504] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } }
I20260806 18:37:51.167259 3170 leader_election.cc:288] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [CANDIDATE]: Term 1 election: Requested vote from peers 06d3d04d41944752bae1c881c3a2d526 (127.18.125.129:34633), 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339)
I20260806 18:37:51.167646 2825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858322b8e014d4fb35b5f10a06ff11b" candidate_uuid: "6df4b687354d41049a4310cc028718cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06d3d04d41944752bae1c881c3a2d526"
I20260806 18:37:51.167637 3099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858322b8e014d4fb35b5f10a06ff11b" candidate_uuid: "6df4b687354d41049a4310cc028718cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9879ac85d2854485bee53d9a65de24e4"
I20260806 18:37:51.167755 3099 raft_consensus.cc:3037] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:51.167790 2825 raft_consensus.cc:3037] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:37:51.168915 3099 raft_consensus.cc:2454] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6df4b687354d41049a4310cc028718cd in term 1.
I20260806 18:37:51.168985 2825 raft_consensus.cc:2454] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6df4b687354d41049a4310cc028718cd in term 1.
I20260806 18:37:51.169262 2892 leader_election.cc:302] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6df4b687354d41049a4310cc028718cd, 9879ac85d2854485bee53d9a65de24e4; no voters:
I20260806 18:37:51.169476 3170 raft_consensus.cc:2781] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:37:51.169660 3170 raft_consensus.cc:686] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 1 LEADER]: Becoming Leader. State: Replica: 6df4b687354d41049a4310cc028718cd, State: Running, Role: LEADER
I20260806 18:37:51.170008 3170 consensus_queue.cc:237] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } }
I20260806 18:37:51.173241 2677 catalog_manager.cc:5495] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 6df4b687354d41049a4310cc028718cd (127.18.125.130). New cstate: current_term: 1 leader_uuid: "6df4b687354d41049a4310cc028718cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } health_report { overall_health: HEALTHY } } }
I20260806 18:37:51.202268 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:51.198884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49844 (local address 127.18.125.130:37439)
0806 18:37:51.199022 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:37:51.199026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:37:51.199040 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:37:51.199132 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:37:51.199135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:37:51.199192 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:37:51.199274 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:37:51.199505 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:51.200007 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:51.200857 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:37:51.201069 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:37:51.201146 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:37:51.201427 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:37:51.201853 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:37:51.201951 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:37:51.202039 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 18:37:51.202089 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:37:51.208716 3099 raft_consensus.cc:1261] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4 [term 1 FOLLOWER]: Refusing update from remote peer 6df4b687354d41049a4310cc028718cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:51.208846 2825 raft_consensus.cc:1261] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Refusing update from remote peer 6df4b687354d41049a4310cc028718cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:37:51.209545 3173 consensus_queue.cc:1035] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:51.209748 3170 consensus_queue.cc:1035] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:37:51.216965 3176 mvcc.cc:205] Tried to move back new op lower bound from 7315625866062348288 to 7315625865915023360. Current Snapshot: MvccSnapshot[applied={T|T < 7315625866062348288}]
I20260806 18:37:51.219029 3178 mvcc.cc:205] Tried to move back new op lower bound from 7315625866062348288 to 7315625865915023360. Current Snapshot: MvccSnapshot[applied={T|T < 7315625866062348288}]
I20260806 18:37:51.219226 3177 mvcc.cc:205] Tried to move back new op lower bound from 7315625866062348288 to 7315625865915023360. Current Snapshot: MvccSnapshot[applied={T|T < 7315625866062348288}]
I20260806 18:37:51.227332 2679 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:51.227504 2679 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:37:51.230100 2679 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Sending DeleteTablet for 3 replicas of tablet b858322b8e014d4fb35b5f10a06ff11b
I20260806 18:37:51.231084 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 2742
I20260806 18:37:51.230943 3077 tablet_service.cc:1514] Processing DeleteTablet for tablet b858322b8e014d4fb35b5f10a06ff11b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:51 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:52482
I20260806 18:37:51.231007 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet b858322b8e014d4fb35b5f10a06ff11b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:51 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:43596
I20260806 18:37:51.231423 2940 tablet_service.cc:1514] Processing DeleteTablet for tablet b858322b8e014d4fb35b5f10a06ff11b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:51 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:49834
I20260806 18:37:51.232451 3186 tablet_replica.cc:330] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: stopping tablet replica
I20260806 18:37:51.232873 3186 raft_consensus.cc:2229] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:37:51.233210 3185 tablet_replica.cc:330] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: stopping tablet replica
I20260806 18:37:51.233888 3185 raft_consensus.cc:2229] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:37:51.234159 3186 raft_consensus.cc:2258] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:51.234433 3185 raft_consensus.cc:2258] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:37:51.236603 3186 ts_tablet_manager.cc:1905] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:51.236974 3185 ts_tablet_manager.cc:1905] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:37:51.240656 3186 ts_tablet_manager.cc:1918] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:51.240906 3186 log.cc:1196] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/b858322b8e014d4fb35b5f10a06ff11b
I20260806 18:37:51.241117 3185 ts_tablet_manager.cc:1918] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:37:51.241259 3185 log.cc:1196] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/b858322b8e014d4fb35b5f10a06ff11b
I20260806 18:37:51.241514 3186 ts_tablet_manager.cc:1939] T b858322b8e014d4fb35b5f10a06ff11b P 6df4b687354d41049a4310cc028718cd: Deleting consensus metadata
I20260806 18:37:51.241622 3185 ts_tablet_manager.cc:1939] T b858322b8e014d4fb35b5f10a06ff11b P 9879ac85d2854485bee53d9a65de24e4: Deleting consensus metadata
I20260806 18:37:51.243129 2663 catalog_manager.cc:4841] TS 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339): tablet b858322b8e014d4fb35b5f10a06ff11b (table test-table [id=be5e53d9227843b69f02b621530842cc]) successfully deleted
I20260806 18:37:51.243141 2664 catalog_manager.cc:4841] TS 6df4b687354d41049a4310cc028718cd (127.18.125.130:37439): tablet b858322b8e014d4fb35b5f10a06ff11b (table test-table [id=be5e53d9227843b69f02b621530842cc]) successfully deleted
I20260806 18:37:51.244498 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 2876
W20260806 18:37:51.244509 2664 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:34633 (error 108)
I20260806 18:37:51.246769 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:37:51.246433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:34633 (local address 127.0.0.1:43610)
0806 18:37:51.246608 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 18:37:51.246684 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:34633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260806 18:37:51.246925 2664 catalog_manager.cc:4584] TS 06d3d04d41944752bae1c881c3a2d526 (127.18.125.129:34633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b858322b8e014d4fb35b5f10a06ff11b: Network error: Client connection negotiation failed: client connection to 127.18.125.129:34633: connect: Connection refused (error 111)
I20260806 18:37:51.254331 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 3012
I20260806 18:37:51.261777 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 2647
I20260806 18:38:11.270373 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:39117
--webserver_interface=127.18.125.190
--webserver_port=32887
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:39117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:11.392661 3188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:11.392962 3188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:11.393006 3188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:11.397079 3188 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:38:11.397178 3188 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:38:11.397220 3188 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:38:11.397255 3188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:11.397286 3188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:11.397323 3188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:11.397370 3188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:11.397435 3188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:11.403136 3188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:39117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:39117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=32887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:11.404486 3188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:11.405625 3188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:11.420183 3197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:11.420185 3194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:11.420284 3188 server_base.cc:1034] running on GCE node
W20260806 18:38:11.420219 3195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:11.421195 3188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:11.422441 3188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:11.423612 3188 hybrid_clock.cc:648] HybridClock initialized: now 1786041491423579 us; error 49 us; skew 500 ppm
Aug 06 18:38:11 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041491, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:11.432920 3188 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:38:11.434046 3188 webserver.cc:460] Webserver started at http://127.18.125.190:32887/ using document root <none> and password file <none>
I20260806 18:38:11.434662 3188 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:11.434746 3188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:11.439107 3188 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:38:11.441785 3204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.443329 3188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:38:11.443490 3188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "106f0a802ca94b2db35c7adf7b1d5bcb"
format_stamp: "Formatted at 2026-08-06 18:37:49 on dist-test-slave-8w9v"
server_key: "39fed4bc5f09b54d582f754f2b51e5fc"
server_key_iv: "b20a445f813a70afcb2d9f742b2c2fd5"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.444275 3188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:11.455799 3188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:11.459154 3188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:11.459400 3188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:11.468786 3188 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:39117
I20260806 18:38:11.469386 3257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:39117 every 8 connection(s)
I20260806 18:38:11.470726 3188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:11.476873 3258 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:38:11.479007 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 3188
I20260806 18:38:11.479127 3258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Bootstrap starting.
I20260806 18:38:11.479542 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:34633
--local_ip_for_outbound_sockets=127.18.125.129
--tserver_master_addrs=127.18.125.190:39117
--webserver_port=45079
--webserver_interface=127.18.125.129
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:38:11.499604 3258 log.cc:824] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:11.509954 3258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:38:11.510437 3258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Bootstrap complete.
I20260806 18:38:11.514860 3258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:38:11.515297 3258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106f0a802ca94b2db35c7adf7b1d5bcb, State: Initialized, Role: FOLLOWER
I20260806 18:38:11.515817 3258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:38:11.515987 3258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:11.516081 3258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:11.516230 3258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:38:11.518306 3258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:38:11.518675 3258 leader_election.cc:302] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 106f0a802ca94b2db35c7adf7b1d5bcb; no voters:
I20260806 18:38:11.519038 3258 leader_election.cc:288] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:38:11.519136 3262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:38:11.519325 3262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 2 LEADER]: Becoming Leader. State: Replica: 106f0a802ca94b2db35c7adf7b1d5bcb, State: Running, Role: LEADER
I20260806 18:38:11.519630 3262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:38:11.520402 3258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:11.521137 3263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } } }
I20260806 18:38:11.521298 3263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:11.521654 3264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 106f0a802ca94b2db35c7adf7b1d5bcb. Latest consensus state: current_term: 2 leader_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } } }
I20260806 18:38:11.521821 3264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:11.522168 3270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:11.527793 3270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:11.528605 3270 catalog_manager.cc:1259] Loaded cluster ID: 62e710d393ea4ea29b35839b8c4edd6a
I20260806 18:38:11.528671 3270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:11.531898 3270 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:11.533072 3270 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Loaded TSK: 0
I20260806 18:38:11.548375 3270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Generated new TSK 1
I20260806 18:38:11.548683 3270 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:38:11.615327 3260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:11.615700 3260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:11.615788 3260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:11.620393 3260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:11.620503 3260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:11.620615 3260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:38:11.626664 3260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:34633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=45079
--tserver_master_addrs=127.18.125.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:11.627908 3260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:11.629132 3260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:11.641240 3286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:11.641434 3260 server_base.cc:1034] running on GCE node
W20260806 18:38:11.641263 3285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:11.641782 3288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:11.642371 3260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:11.643173 3260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:11.644366 3260 hybrid_clock.cc:648] HybridClock initialized: now 1786041491644354 us; error 38 us; skew 500 ppm
Aug 06 18:38:11 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041491, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:11.647488 3260 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:38:11.648747 3260 webserver.cc:460] Webserver started at http://127.18.125.129:45079/ using document root <none> and password file <none>
I20260806 18:38:11.649446 3260 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:11.649533 3260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:11.654018 3260 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:38:11.656673 3295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.658139 3260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:11.658283 3260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "06d3d04d41944752bae1c881c3a2d526"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "31952d7532bbc63aa2c3363ea1071c5d"
server_key_iv: "3eb62e1ce5dd0682f1df32c8f09c21c5"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.658928 3260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:11.683439 3260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:11.687776 3260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:11.688027 3260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:11.688979 3260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:11.690303 3302 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:38:11.694875 3260 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:38:11.694964 3260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 18:38:11.695024 3260 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:38:11.696976 3260 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:38:11.697027 3260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 18:38:11.697324 3302 tablet_bootstrap.cc:492] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Bootstrap starting.
I20260806 18:38:11.708994 3260 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:34633
I20260806 18:38:11.709091 3410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:34633 every 8 connection(s)
I20260806 18:38:11.710127 3260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:11.717931 3302 log.cc:824] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:11.718385 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 3260
I20260806 18:38:11.718971 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:37439
--local_ip_for_outbound_sockets=127.18.125.130
--tserver_master_addrs=127.18.125.190:39117
--webserver_port=32865
--webserver_interface=127.18.125.130
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:38:11 dist-test-slave-8w9v krb5kdc[2634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041491, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:11.726845 3302 tablet_bootstrap.cc:492] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:38:11.727288 3302 tablet_bootstrap.cc:492] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Bootstrap complete.
I20260806 18:38:11.728955 3302 ts_tablet_manager.cc:1397] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.012s
I20260806 18:38:11.731559 3414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.712416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:53703)
0806 18:38:11.713222 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.713283 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:11.714579 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:11.715780 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:11.715789 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:11.716374 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:11.717053 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.717076 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.718654 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.718658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.720178 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.720196 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.721405 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.721413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.721645 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.722426 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:11.722452 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:11.724513 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:11.727550 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:11.727556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:11.727570 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:11.727976 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:11.728523 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:11.728529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:11.728535 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:11.728734 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:11.729641 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:11.729647 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:11.729955 (+ 308us) client_negotiation.cc:769] Sending connection context
0806 18:38:11.730158 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 18:38:11.730528 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":160,"threads_started":1}
I20260806 18:38:11.731765 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.712417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:53703 (local address 127.18.125.190:39117)
0806 18:38:11.712876 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.712897 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.713427 (+ 530us) server_negotiation.cc:407] Connection header received
0806 18:38:11.715026 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.715037 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.715350 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.715626 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:11.717251 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.718424 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.720362 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.721216 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.721398 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.724799 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:11.724808 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:11.724830 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:11.724883 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:11.727398 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:11.728169 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:11.728177 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:11.728184 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:11.728387 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:11.728867 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:11.728873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:11.728878 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:11.729491 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:11.729660 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.730374 (+ 714us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.730751 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":128,"threads_started":1}
I20260806 18:38:11.732089 3302 raft_consensus.cc:348] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } }
I20260806 18:38:11.732686 3302 raft_consensus.cc:729] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d3d04d41944752bae1c881c3a2d526, State: Initialized, Role: FOLLOWER
I20260806 18:38:11.733203 3302 consensus_queue.cc:260] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } }
I20260806 18:38:11.734309 3302 ts_tablet_manager.cc:1428] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260806 18:38:11.734876 3411 heartbeater.cc:347] Connected to a master server at 127.18.125.190:39117
I20260806 18:38:11.735140 3411 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:11.735724 3411 heartbeater.cc:511] Master 127.18.125.190:39117 requested a full tablet report, sending...
I20260806 18:38:11.738721 3222 ts_manager.cc:194] Registered new tserver with Master: 06d3d04d41944752bae1c881c3a2d526 (127.18.125.129:34633)
I20260806 18:38:11.743145 3222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:53703
I20260806 18:38:11.747061 3411 heartbeater.cc:503] Master 127.18.125.190:39117 was elected leader, sending a full tablet report...
Aug 06 18:38:11 dist-test-slave-8w9v krb5kdc[2634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041491, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:38:11.758090 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.742501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:34633 (local address 127.0.0.1:37888)
0806 18:38:11.743334 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.743385 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:11.743576 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:11.744410 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:11.744419 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:11.744938 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:11.745125 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.745137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.748265 (+ 3128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.748272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.749419 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.749433 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.750276 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.750281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.750438 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.751126 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:11.751142 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:11.752966 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:11.756083 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:11.756094 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:11.756099 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:11.756450 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:11.756822 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:11.756824 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:11.756826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:11.756910 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:11.757499 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:11.757504 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:11.757624 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 18:38:11.757741 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 18:38:11.757932 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":124,"threads_started":1}
I20260806 18:38:11.758237 3419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.742574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37888 (local address 127.18.125.129:34633)
0806 18:38:11.743048 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.743060 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.743502 (+ 442us) server_negotiation.cc:407] Connection header received
0806 18:38:11.743790 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.743798 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.744051 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.744206 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:11.745693 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.748031 (+ 2338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.749588 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.750120 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.750242 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.753155 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:11.753167 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:11.753182 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:11.753212 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:11.755890 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:11.756596 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:11.756603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:11.756605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:11.756710 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:11.757014 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:11.757018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:11.757019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:11.757371 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:11.757510 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.757877 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.758066 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":117,"threads_started":1}
I20260806 18:38:11.759341 3344 tablet_service.cc:1514] Processing DeleteTablet for tablet b858322b8e014d4fb35b5f10a06ff11b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:37:51 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:37888
I20260806 18:38:11.760447 3420 tablet_replica.cc:330] stopping tablet replica
I20260806 18:38:11.760782 3420 raft_consensus.cc:2229] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:11.761056 3420 raft_consensus.cc:2258] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:11.762118 3420 ts_tablet_manager.cc:1905] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:11.765977 3420 ts_tablet_manager.cc:1918] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:11.766068 3420 log.cc:1196] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal/wals/b858322b8e014d4fb35b5f10a06ff11b
I20260806 18:38:11.766360 3420 ts_tablet_manager.cc:1939] T b858322b8e014d4fb35b5f10a06ff11b P 06d3d04d41944752bae1c881c3a2d526: Deleting consensus metadata
I20260806 18:38:11.768551 3206 catalog_manager.cc:4841] TS 06d3d04d41944752bae1c881c3a2d526 (127.18.125.129:34633): tablet b858322b8e014d4fb35b5f10a06ff11b (table test-table [id=be5e53d9227843b69f02b621530842cc]) successfully deleted
W20260806 18:38:11.856740 3416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:11.857067 3416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:11.857165 3416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:11.861397 3416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:11.861510 3416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:11.861608 3416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:38:11.867961 3416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:37439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=32865
--tserver_master_addrs=127.18.125.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:11.869303 3416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:11.870450 3416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:11.880820 3428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:11.880839 3426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:11.880827 3425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:11.881069 3416 server_base.cc:1034] running on GCE node
I20260806 18:38:11.881632 3416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:11.882423 3416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:11.883615 3416 hybrid_clock.cc:648] HybridClock initialized: now 1786041491883589 us; error 51 us; skew 500 ppm
Aug 06 18:38:11 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041491, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:11.886502 3416 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:38:11.887427 3416 webserver.cc:460] Webserver started at http://127.18.125.130:32865/ using document root <none> and password file <none>
I20260806 18:38:11.887979 3416 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:11.888049 3416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:11.892000 3416 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:38:11.894164 3435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.895455 3416 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:11.895604 3416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "6df4b687354d41049a4310cc028718cd"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "5fec30685d9812b520f33558464f2ac7"
server_key_iv: "84fac2e17495733576667738a4352fc0"
server_key_version: "encryptionkey@0"
I20260806 18:38:11.896250 3416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:11.910884 3416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:11.914125 3416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:11.914346 3416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:11.915004 3416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:11.915927 3416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:11.915992 3416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.916066 3416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:11.916106 3416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:11.926358 3416 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:37439
I20260806 18:38:11.926383 3548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:37439 every 8 connection(s)
I20260806 18:38:11.927424 3416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:11.928086 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 3416
I20260806 18:38:11.928591 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:33339
--local_ip_for_outbound_sockets=127.18.125.131
--tserver_master_addrs=127.18.125.190:39117
--webserver_port=37163
--webserver_interface=127.18.125.131
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:38:11 dist-test-slave-8w9v krb5kdc[2634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041491, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:11.943051 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.929662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:49645 (local address 127.18.125.190:39117)
0806 18:38:11.929865 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:11.929872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:11.930701 (+ 829us) server_negotiation.cc:407] Connection header received
0806 18:38:11.932100 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:11.932107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:11.932207 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:11.932330 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:11.933530 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.934474 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.935631 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.936100 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.936287 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.938916 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:11.938922 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:11.938927 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:11.938967 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:11.940846 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:11.941408 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:11.941412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:11.941414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:11.941509 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:11.941842 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:11.941845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:11.941847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:11.942082 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:11.942169 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:11.942701 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 18:38:11.942848 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:38:11.943910 3552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:11.930044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:49645)
0806 18:38:11.930527 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:11.930581 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:11.931866 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:11.932513 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:11.932520 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:11.932876 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:11.933366 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.933380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.934613 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.934616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.935497 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:11.935506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:11.936314 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:11.936321 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:11.936565 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:11.937213 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:11.937230 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:11.938723 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:11.940987 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:11.940992 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:11.941006 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:11.941296 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:11.941611 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:11.941613 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:11.941614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:11.941755 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:11.942188 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:11.942193 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:11.942468 (+ 275us) client_negotiation.cc:769] Sending connection context
0806 18:38:11.942634 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 18:38:11.942909 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":135,"threads_started":1}
I20260806 18:38:11.945214 3549 heartbeater.cc:347] Connected to a master server at 127.18.125.190:39117
I20260806 18:38:11.945497 3549 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:11.946098 3549 heartbeater.cc:511] Master 127.18.125.190:39117 requested a full tablet report, sending...
I20260806 18:38:11.947352 3222 ts_manager.cc:194] Registered new tserver with Master: 6df4b687354d41049a4310cc028718cd (127.18.125.130:37439)
I20260806 18:38:11.948055 3222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:49645
W20260806 18:38:12.052383 3551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:12.052668 3551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:12.052762 3551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:12.057093 3551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:12.057163 3551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:12.057255 3551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:38:12.062862 3551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:33339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=37163
--tserver_master_addrs=127.18.125.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:12.064462 3551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:12.065610 3551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:12.076033 3558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:12.076090 3561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:12.076004 3559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:12.076350 3551 server_base.cc:1034] running on GCE node
I20260806 18:38:12.077001 3551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:12.077682 3551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:12.078894 3551 hybrid_clock.cc:648] HybridClock initialized: now 1786041492078880 us; error 26 us; skew 500 ppm
Aug 06 18:38:12 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041492, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:12.081890 3551 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:38:12.082907 3551 webserver.cc:460] Webserver started at http://127.18.125.131:37163/ using document root <none> and password file <none>
I20260806 18:38:12.083526 3551 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:12.083657 3551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:12.087935 3551 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:12.090385 3568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:12.091784 3551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:38:12.091960 3551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "9879ac85d2854485bee53d9a65de24e4"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "60b5d89fec6a3ca050c2e05fd3954fef"
server_key_iv: "17f1e55c23cc626e186d74a1aa3c5948"
server_key_version: "encryptionkey@0"
I20260806 18:38:12.092690 3551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:12.112268 3551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:12.115618 3551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:12.115849 3551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:12.116582 3551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:12.117475 3551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:12.117548 3551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:12.117621 3551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:12.117671 3551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:12.127621 3551 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:33339
I20260806 18:38:12.127646 3682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:33339 every 8 connection(s)
I20260806 18:38:12.128695 3551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:12.136580 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 3551
Aug 06 18:38:12 dist-test-slave-8w9v krb5kdc[2634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041492, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:12.145465 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.130744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:55761 (local address 127.18.125.190:39117)
0806 18:38:12.130911 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:12.130916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:12.131840 (+ 924us) server_negotiation.cc:407] Connection header received
0806 18:38:12.133217 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:12.133220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:12.133268 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:12.133353 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:12.134536 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.135111 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.136473 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.136892 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.137193 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.140859 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:12.140862 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:12.140864 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:12.140896 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:12.142569 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:12.143277 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:12.143281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:12.143282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:12.143361 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:12.143752 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:12.143757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:12.143758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:12.143996 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:12.144078 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:12.145078 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0806 18:38:12.145234 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:38:12.146461 3685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.131177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:55761)
0806 18:38:12.131668 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:12.131710 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:12.133006 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:12.133515 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:12.133521 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:12.133887 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:12.134384 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:12.134398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.135236 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.135257 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:12.136223 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:12.136233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.137223 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.137229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:12.137490 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.138369 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:12.138419 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:12.140671 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:12.142696 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:12.142704 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:12.142723 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:12.143141 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:12.143462 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:12.143467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:12.143473 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:12.143645 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:12.144164 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:12.144173 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:12.144543 (+ 370us) client_negotiation.cc:769] Sending connection context
0806 18:38:12.144949 (+ 406us) client_negotiation.cc:240] Negotiation successful
0806 18:38:12.145336 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":121,"threads_started":1}
I20260806 18:38:12.147917 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.137275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.18.125.190:39117)
0806 18:38:12.137571 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:12.137575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:12.137729 (+ 154us) server_negotiation.cc:407] Connection header received
0806 18:38:12.137897 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:12.137902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:12.137953 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:12.138144 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:12.139200 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.140218 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.141355 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.141771 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.141869 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.143548 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:12.143551 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:12.143553 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:12.143594 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:12.145754 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:12.146249 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:12.146251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:12.146252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:12.146331 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:12.146971 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:12.146975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:12.146976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:12.147208 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:12.147489 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:12.147622 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 18:38:12.147750 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":108,"threads_started":1}
I20260806 18:38:12.147991 3683 heartbeater.cc:347] Connected to a master server at 127.18.125.190:39117
I20260806 18:38:12.148298 3683 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:12.148988 3683 heartbeater.cc:511] Master 127.18.125.190:39117 requested a full tablet report, sending...
I20260806 18:38:12.150197 3222 ts_manager.cc:194] Registered new tserver with Master: 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339)
I20260806 18:38:12.150839 3222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:55761
I20260806 18:38:12.162313 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:12.166908 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.163033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41086 (local address 127.18.125.190:39117)
0806 18:38:12.163178 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:12.163182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:12.163433 (+ 251us) server_negotiation.cc:407] Connection header received
0806 18:38:12.163644 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:12.163647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:12.163761 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:12.163858 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:12.164109 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.164708 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.165605 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.165850 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.165936 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.166345 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:12.166504 (+ 159us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 18:38:12.166777 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41086: token expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 18:38:12.167013 3687 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41086: token expired
W20260806 18:38:12.167367 3149 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.125.190:39117)
I20260806 18:38:12.175689 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.167775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41100 (local address 127.18.125.190:39117)
0806 18:38:12.167901 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:12.167909 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:12.167938 (+ 29us) server_negotiation.cc:407] Connection header received
0806 18:38:12.168043 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:12.168045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:12.168083 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:12.168228 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:12.168827 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.169309 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.170101 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.170537 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.170642 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.172318 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:12.172322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:12.172324 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:12.172348 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:12.174015 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:12.174429 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:12.174432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:12.174433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:12.174511 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:12.174844 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:12.174846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:12.174847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:12.175020 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:12.175123 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:12.175419 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 18:38:12.175544 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:38:12.178414 3222 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:12.181910 3222 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:12.190276 3344 tablet_service.cc:1467] Processing CreateTablet for tablet c85b8ecc100942188b324033c75724bc (DEFAULT_TABLE table=test-table [id=da6e3574a5ac4d619c2fdb0c68196d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:12.191056 3344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85b8ecc100942188b324033c75724bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:12.194186 3302 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: Bootstrap starting.
I20260806 18:38:12.195080 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.188862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:37439 (local address 127.0.0.1:38620)
0806 18:38:12.189174 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:12.189191 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:12.189270 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:12.190350 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:12.190353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:12.190371 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:12.190490 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:12.190500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.191873 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.191877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:12.193358 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:12.193370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.194559 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.194563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:12.194687 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.194747 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 18:38:12.194832 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:38:12.194929 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":102}
I20260806 18:38:12.195309 3302 tablet_bootstrap.cc:654] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:12.195489 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.189204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:33339 (local address 127.0.0.1:43500)
0806 18:38:12.189539 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:12.189554 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:12.189642 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:12.190480 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:12.190485 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:12.190507 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:12.190622 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:12.190629 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.192169 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.192172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:12.193367 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:12.193373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.194845 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.194848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:12.195140 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.195178 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:38:12.195254 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:38:12.195331 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":27,"thread_start_us":107,"threads_started":1}
I20260806 18:38:12.195878 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.189049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.18.125.130:37439)
0806 18:38:12.189855 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:12.189869 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:12.189896 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:38:12.189983 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:12.189988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:12.190175 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:12.190349 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:12.190673 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.191732 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.193542 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.194402 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.195303 (+ 901us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.195483 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:12.195554 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 18:38:12.195696 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":110,"threads_started":1}
I20260806 18:38:12.197000 3302 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: No bootstrap required, opened a new log
I20260806 18:38:12.197034 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.189592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.18.125.131:33339)
0806 18:38:12.190004 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:12.190016 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:12.190039 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:38:12.190115 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:12.190120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:12.190309 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:12.190780 (+ 471us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:12.190969 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.192039 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.193514 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.194712 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.195867 (+ 1155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.196466 (+ 599us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:12.196683 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 18:38:12.196852 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":160,"threads_started":1}
I20260806 18:38:12.197109 3302 ts_tablet_manager.cc:1397] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 18:38:12.197664 3302 raft_consensus.cc:348] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:12.197808 3302 raft_consensus.cc:374] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:12.197860 3302 raft_consensus.cc:729] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d3d04d41944752bae1c881c3a2d526, State: Initialized, Role: FOLLOWER
I20260806 18:38:12.198009 3302 consensus_queue.cc:260] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:12.198540 3302 ts_tablet_manager.cc:1428] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260806 18:38:12.199488 3483 tablet_service.cc:1467] Processing CreateTablet for tablet c85b8ecc100942188b324033c75724bc (DEFAULT_TABLE table=test-table [id=da6e3574a5ac4d619c2fdb0c68196d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:12.200289 3616 tablet_service.cc:1467] Processing CreateTablet for tablet c85b8ecc100942188b324033c75724bc (DEFAULT_TABLE table=test-table [id=da6e3574a5ac4d619c2fdb0c68196d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:12.201604 3616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85b8ecc100942188b324033c75724bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:12.202204 3483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85b8ecc100942188b324033c75724bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:12.210295 3696 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Bootstrap starting.
I20260806 18:38:12.211617 3697 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Bootstrap starting.
W20260806 18:38:12.212360 3412 tablet.cc:2369] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:12.213806 3697 tablet_bootstrap.cc:654] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:12.214032 3696 tablet_bootstrap.cc:654] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:12.214818 3696 log.cc:824] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:12.214963 3697 log.cc:824] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:12.217280 3696 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: No bootstrap required, opened a new log
I20260806 18:38:12.217504 3697 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: No bootstrap required, opened a new log
I20260806 18:38:12.217542 3696 ts_tablet_manager.cc:1397] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260806 18:38:12.217681 3697 ts_tablet_manager.cc:1397] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:38:12.220727 3696 raft_consensus.cc:348] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:12.220752 3697 raft_consensus.cc:348] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:12.221021 3697 raft_consensus.cc:374] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:12.221009 3696 raft_consensus.cc:374] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:12.221081 3697 raft_consensus.cc:729] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9879ac85d2854485bee53d9a65de24e4, State: Initialized, Role: FOLLOWER
I20260806 18:38:12.221089 3696 raft_consensus.cc:729] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df4b687354d41049a4310cc028718cd, State: Initialized, Role: FOLLOWER
I20260806 18:38:12.221563 3697 consensus_queue.cc:260] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:12.221562 3696 consensus_queue.cc:260] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:12.222292 3549 heartbeater.cc:503] Master 127.18.125.190:39117 was elected leader, sending a full tablet report...
I20260806 18:38:12.222325 3683 heartbeater.cc:503] Master 127.18.125.190:39117 was elected leader, sending a full tablet report...
I20260806 18:38:12.222666 3696 ts_tablet_manager.cc:1428] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:38:12.223613 3697 ts_tablet_manager.cc:1428] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:38:12.378116 3417 raft_consensus.cc:482] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:12.378353 3417 raft_consensus.cc:504] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:12.379622 3417 leader_election.cc:288] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6df4b687354d41049a4310cc028718cd (127.18.125.130:37439), 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339)
W20260806 18:38:12.379781 3684 tablet.cc:2369] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:12.384020 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.380398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:53367 (local address 127.18.125.131:33339)
0806 18:38:12.380537 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:12.380543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:12.380644 (+ 101us) server_negotiation.cc:407] Connection header received
0806 18:38:12.380820 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:12.380823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:12.380865 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:12.380976 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:12.381283 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.381769 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.382927 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.383368 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.383504 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.383572 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:12.383779 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 18:38:12.383868 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:38:12.384027 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.380289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:59511 (local address 127.18.125.130:37439)
0806 18:38:12.380534 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:12.380539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:12.380559 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:38:12.380770 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:12.380775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:12.380830 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:12.380894 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:12.381248 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.381754 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.382981 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.383396 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.383535 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.383588 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:12.383832 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 18:38:12.383919 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260806 18:38:12.384045 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.379969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:33339 (local address 127.18.125.129:53367)
0806 18:38:12.380520 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:12.380535 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:12.380664 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:12.381021 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:12.381025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:12.381053 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:12.381127 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:12.381131 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.381917 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.381920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:12.382786 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:12.382795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.383511 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.383515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:12.383607 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.383663 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 18:38:12.383727 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:38:12.383831 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":77,"thread_start_us":109,"threads_started":1}
I20260806 18:38:12.384089 3702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.379969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:37439 (local address 127.18.125.129:59511)
0806 18:38:12.380428 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:12.380447 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:12.380587 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:12.380946 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:12.380950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:12.380975 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:12.381115 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:12.381122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.381885 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.381889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:12.382815 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:12.382827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.383559 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.383564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:12.383665 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.383685 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:38:12.383800 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 18:38:12.383875 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":60,"thread_start_us":122,"threads_started":1}
I20260806 18:38:12.384953 3503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85b8ecc100942188b324033c75724bc" candidate_uuid: "06d3d04d41944752bae1c881c3a2d526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6df4b687354d41049a4310cc028718cd" is_pre_election: true
I20260806 18:38:12.384990 3636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85b8ecc100942188b324033c75724bc" candidate_uuid: "06d3d04d41944752bae1c881c3a2d526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9879ac85d2854485bee53d9a65de24e4" is_pre_election: true
I20260806 18:38:12.385305 3636 raft_consensus.cc:2454] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d3d04d41944752bae1c881c3a2d526 in term 0.
I20260806 18:38:12.385304 3503 raft_consensus.cc:2454] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d3d04d41944752bae1c881c3a2d526 in term 0.
I20260806 18:38:12.385901 3297 leader_election.cc:302] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 06d3d04d41944752bae1c881c3a2d526, 6df4b687354d41049a4310cc028718cd, 9879ac85d2854485bee53d9a65de24e4; no voters:
I20260806 18:38:12.386267 3417 raft_consensus.cc:2781] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:12.386361 3417 raft_consensus.cc:482] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:12.386415 3417 raft_consensus.cc:3037] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:12.387485 3417 raft_consensus.cc:504] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:12.387856 3417 leader_election.cc:288] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [CANDIDATE]: Term 1 election: Requested vote from peers 6df4b687354d41049a4310cc028718cd (127.18.125.130:37439), 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339)
I20260806 18:38:12.388265 3503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85b8ecc100942188b324033c75724bc" candidate_uuid: "06d3d04d41944752bae1c881c3a2d526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6df4b687354d41049a4310cc028718cd"
I20260806 18:38:12.388408 3636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85b8ecc100942188b324033c75724bc" candidate_uuid: "06d3d04d41944752bae1c881c3a2d526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9879ac85d2854485bee53d9a65de24e4"
I20260806 18:38:12.388422 3503 raft_consensus.cc:3037] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:12.388549 3636 raft_consensus.cc:3037] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:12.389884 3503 raft_consensus.cc:2454] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d3d04d41944752bae1c881c3a2d526 in term 1.
I20260806 18:38:12.389884 3636 raft_consensus.cc:2454] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d3d04d41944752bae1c881c3a2d526 in term 1.
I20260806 18:38:12.390218 3297 leader_election.cc:302] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d3d04d41944752bae1c881c3a2d526, 6df4b687354d41049a4310cc028718cd; no voters:
I20260806 18:38:12.390403 3417 raft_consensus.cc:2781] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:12.390619 3417 raft_consensus.cc:686] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 1 LEADER]: Becoming Leader. State: Replica: 06d3d04d41944752bae1c881c3a2d526, State: Running, Role: LEADER
I20260806 18:38:12.390933 3417 consensus_queue.cc:237] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:12.394145 3222 catalog_manager.cc:5495] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d3d04d41944752bae1c881c3a2d526 (127.18.125.129). New cstate: current_term: 1 leader_uuid: "06d3d04d41944752bae1c881c3a2d526" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } health_report { overall_health: HEALTHY } } }
W20260806 18:38:12.428778 3550 tablet.cc:2369] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:12.440855 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.437086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37902 (local address 127.18.125.129:34633)
0806 18:38:12.437443 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:12.437451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:12.437464 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:38:12.437541 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:12.437543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:12.437598 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:12.437699 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:12.437962 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.438488 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.439316 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:12.439545 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:12.439616 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:12.439863 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:12.440399 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:12.440510 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:12.440602 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 18:38:12.440654 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":117,"threads_started":1}
I20260806 18:38:12.445336 3503 raft_consensus.cc:1261] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Refusing update from remote peer 06d3d04d41944752bae1c881c3a2d526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:12.445336 3636 raft_consensus.cc:1261] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 1 FOLLOWER]: Refusing update from remote peer 06d3d04d41944752bae1c881c3a2d526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:12.446131 3417 consensus_queue.cc:1035] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:12.446388 3704 consensus_queue.cc:1035] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:12.454422 3708 mvcc.cc:205] Tried to move back new op lower bound from 7315625953048383488 to 7315625952836182016. Current Snapshot: MvccSnapshot[applied={T|T < 7315625953048383488}]
I20260806 18:38:12.456362 3709 mvcc.cc:205] Tried to move back new op lower bound from 7315625953048383488 to 7315625952836182016. Current Snapshot: MvccSnapshot[applied={T|T < 7315625953048383488}]
I20260806 18:38:12.457598 3710 mvcc.cc:205] Tried to move back new op lower bound from 7315625953048383488 to 7315625952836182016. Current Snapshot: MvccSnapshot[applied={T|T < 7315625953048383488}]
I20260806 18:38:12.465739 3222 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:12.465993 3222 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:12.469877 3222 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Sending DeleteTablet for 3 replicas of tablet c85b8ecc100942188b324033c75724bc
I20260806 18:38:12.470814 3616 tablet_service.cc:1514] Processing DeleteTablet for tablet c85b8ecc100942188b324033c75724bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:12 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:43500
I20260806 18:38:12.470816 3483 tablet_service.cc:1514] Processing DeleteTablet for tablet c85b8ecc100942188b324033c75724bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:12 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:38620
I20260806 18:38:12.471642 3717 tablet_replica.cc:330] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: stopping tablet replica
I20260806 18:38:12.472209 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 3260
I20260806 18:38:12.472282 3344 tablet_service.cc:1514] Processing DeleteTablet for tablet c85b8ecc100942188b324033c75724bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:12 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:37888
I20260806 18:38:12.474073 3718 tablet_replica.cc:330] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: stopping tablet replica
I20260806 18:38:12.474535 3718 raft_consensus.cc:2229] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:12.474948 3718 raft_consensus.cc:2258] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:12.477046 3718 ts_tablet_manager.cc:1905] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:12.479754 3718 ts_tablet_manager.cc:1918] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:12.479887 3718 log.cc:1196] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/c85b8ecc100942188b324033c75724bc
I20260806 18:38:12.480309 3718 ts_tablet_manager.cc:1939] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Deleting consensus metadata
I20260806 18:38:12.480561 3717 raft_consensus.cc:2229] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:12.481024 3717 raft_consensus.cc:2258] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:12.481938 3205 catalog_manager.cc:4841] TS 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339): tablet c85b8ecc100942188b324033c75724bc (table test-table [id=da6e3574a5ac4d619c2fdb0c68196d89]) successfully deleted
I20260806 18:38:12.482885 3717 ts_tablet_manager.cc:1905] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:38:12.482965 3206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:34633 (error 108)
I20260806 18:38:12.483318 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 3416
I20260806 18:38:12.484280 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.483873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:34633 (local address 127.0.0.1:37912)
0806 18:38:12.484078 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:12.484176 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:34633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260806 18:38:12.484458 3206 catalog_manager.cc:4584] TS 06d3d04d41944752bae1c881c3a2d526 (127.18.125.129:34633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c85b8ecc100942188b324033c75724bc: Network error: Client connection negotiation failed: client connection to 127.18.125.129:34633: connect: Connection refused (error 111)
I20260806 18:38:12.493316 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 3551
I20260806 18:38:12.497036 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:12.493630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:37439 (local address 127.0.0.1:38636)
0806 18:38:12.496845 (+ 3215us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:12.496926 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.130:37439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3109}
I20260806 18:38:12.502723 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 3188
I20260806 18:38:32.511461 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:39117
--webserver_interface=127.18.125.190
--webserver_port=32887
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:39117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:32.627702 3720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:32.628062 3720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:32.628221 3720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:32.632306 3720 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:38:32.632406 3720 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:38:32.632433 3720 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:38:32.632458 3720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:32.632503 3720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:32.632561 3720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:32.632599 3720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:32.632638 3720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:32.638155 3720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:39117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:39117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=32887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:32.639384 3720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:32.640498 3720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:38:32.649554 3720 server_base.cc:1034] running on GCE node
W20260806 18:38:32.649554 3728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:32.649613 3726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:32.649613 3725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:32.650471 3720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:32.651682 3720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:32.652882 3720 hybrid_clock.cc:648] HybridClock initialized: now 1786041512652863 us; error 34 us; skew 500 ppm
Aug 06 18:38:32 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041512, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:32.660179 3720 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:38:32.661270 3720 webserver.cc:460] Webserver started at http://127.18.125.190:32887/ using document root <none> and password file <none>
I20260806 18:38:32.661896 3720 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:32.661995 3720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:32.666110 3720 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:38:32.668403 3735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:32.670089 3720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:32.670261 3720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "106f0a802ca94b2db35c7adf7b1d5bcb"
format_stamp: "Formatted at 2026-08-06 18:37:49 on dist-test-slave-8w9v"
server_key: "39fed4bc5f09b54d582f754f2b51e5fc"
server_key_iv: "b20a445f813a70afcb2d9f742b2c2fd5"
server_key_version: "encryptionkey@0"
I20260806 18:38:32.670861 3720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:32.685555 3720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:32.689060 3720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:32.689280 3720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:32.697130 3720 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:39117
I20260806 18:38:32.697456 3789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:39117 every 8 connection(s)
I20260806 18:38:32.698328 3720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:32.699275 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 3720
I20260806 18:38:32.699756 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:34633
--local_ip_for_outbound_sockets=127.18.125.129
--tserver_master_addrs=127.18.125.190:39117
--webserver_port=45079
--webserver_interface=127.18.125.129
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:38:32.703397 3790 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:38:32.705399 3790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Bootstrap starting.
I20260806 18:38:32.724988 3790 log.cc:824] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:32.736603 3790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:38:32.737044 3790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Bootstrap complete.
I20260806 18:38:32.741453 3790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:38:32.741940 3790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106f0a802ca94b2db35c7adf7b1d5bcb, State: Initialized, Role: FOLLOWER
I20260806 18:38:32.742470 3790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:38:32.742628 3790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:32.742724 3790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:32.742858 3790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:38:32.745236 3790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:38:32.745602 3790 leader_election.cc:302] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 106f0a802ca94b2db35c7adf7b1d5bcb; no voters:
I20260806 18:38:32.745978 3790 leader_election.cc:288] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 18:38:32.746080 3795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:38:32.746348 3795 raft_consensus.cc:686] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [term 3 LEADER]: Becoming Leader. State: Replica: 106f0a802ca94b2db35c7adf7b1d5bcb, State: Running, Role: LEADER
I20260806 18:38:32.746668 3795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } }
I20260806 18:38:32.747354 3790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:32.747715 3795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 106f0a802ca94b2db35c7adf7b1d5bcb. Latest consensus state: current_term: 3 leader_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } } }
I20260806 18:38:32.747843 3795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:32.748265 3799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:32.748360 3796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106f0a802ca94b2db35c7adf7b1d5bcb" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 39117 } } }
I20260806 18:38:32.748553 3796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:32.753860 3799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:32.754707 3799 catalog_manager.cc:1259] Loaded cluster ID: 62e710d393ea4ea29b35839b8c4edd6a
I20260806 18:38:32.754770 3799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:32.757418 3799 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:32.758219 3799 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Loaded TSK: 0
I20260806 18:38:32.758317 3799 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 106f0a802ca94b2db35c7adf7b1d5bcb: Loaded TSK: 1
F20260806 18:38:32.759107 3799 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd6f2e319ad google::LogMessage::Fail() at ??:0
@ 0x7fd6f2e3488a google::LogMessage::SendToLog() at ??:0
@ 0x7fd6f2e314de google::LogMessage::Flush() at ??:0
@ 0x7fd6f2e32a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd6f5af518d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fd6f5aedf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fd6fd79c7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fd6fd77699c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fd6fd777178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fd6fd7b8df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fd6fd7d782b std::function<>::operator()() at ??:0
@ 0x7fd6fd776b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fd6fd777f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd6fd774579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fd6fd7b8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd6f50833d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd6f5083c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fd6f50853f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd6f5074512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd6f66bc6db start_thread at ??:0
@ 0x7fd6f192f61f clone at ??:0
W20260806 18:38:32.852543 3793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:32.852804 3793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:32.852851 3793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:32.857491 3793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:32.857546 3793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:32.857609 3793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:38:32.863780 3793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:34633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=45079
--tserver_master_addrs=127.18.125.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:32.865139 3793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:32.866218 3793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:32.878731 3840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:32.878731 3842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:32.879398 3838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:32.880563 3793 server_base.cc:1034] running on GCE node
I20260806 18:38:32.881124 3793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:32.881928 3793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:32.883188 3793 hybrid_clock.cc:648] HybridClock initialized: now 1786041512883151 us; error 59 us; skew 500 ppm
Aug 06 18:38:32 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041512, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:32.886462 3793 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:38:32.887517 3793 webserver.cc:460] Webserver started at http://127.18.125.129:45079/ using document root <none> and password file <none>
I20260806 18:38:32.888207 3793 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:32.888324 3793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:32.892590 3793 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:38:32.895072 3850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:32.896593 3793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:32.896744 3793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "06d3d04d41944752bae1c881c3a2d526"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "31952d7532bbc63aa2c3363ea1071c5d"
server_key_iv: "3eb62e1ce5dd0682f1df32c8f09c21c5"
server_key_version: "encryptionkey@0"
I20260806 18:38:32.897351 3793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:32.919587 3793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:32.923441 3793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:32.923708 3793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:32.924492 3793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:32.925760 3857 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:38:32.931787 3793 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:38:32.931887 3793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 18:38:32.931969 3793 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:38:32.933902 3793 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:38:32.933982 3793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:38:32.934297 3857 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: Bootstrap starting.
I20260806 18:38:32.945677 3793 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:34633
I20260806 18:38:32.945977 3966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:34633 every 8 connection(s)
I20260806 18:38:32.946867 3793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:32.949652 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 3793
I20260806 18:38:32.950165 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:37439
--local_ip_for_outbound_sockets=127.18.125.130
--tserver_master_addrs=127.18.125.190:39117
--webserver_port=32865
--webserver_interface=127.18.125.130
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:38:32.959879 3857 log.cc:824] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:32.972245 3857 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:38:32.972775 3857 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: Bootstrap complete.
I20260806 18:38:32.974475 3857 ts_tablet_manager.cc:1397] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: Time spent bootstrapping tablet: real 0.040s user 0.029s sys 0.008s
I20260806 18:38:32.978376 3857 raft_consensus.cc:348] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:32.979020 3857 raft_consensus.cc:729] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d3d04d41944752bae1c881c3a2d526, State: Initialized, Role: FOLLOWER
I20260806 18:38:32.979627 3857 consensus_queue.cc:260] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:32.980885 3857 ts_tablet_manager.cc:1428] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526: Time spent starting tablet: real 0.006s user 0.003s sys 0.005s
W20260806 18:38:33.051230 3969 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:38:32.949184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:35013)
0806 18:38:32.950773 (+ 1589us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:32.950842 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:32.952182 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:33.050145 (+ 97963us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1309,"thread_start_us":193,"threads_started":1}
W20260806 18:38:33.051888 3967 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:39117 (0 consecutive failures): Network error: Failed to ping master at 127.18.125.190:39117: Client connection negotiation failed: client connection to 127.18.125.190:39117: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:38:33.052840 3969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:33.052472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:48919)
0806 18:38:33.052653 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:33.052748 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 18:38:33.053797 3969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:33.053535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:34081)
0806 18:38:33.053686 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:33.053738 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260806 18:38:33.054109 3967 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:38:33.083250 3970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:33.083608 3970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:33.083701 3970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:33.088101 3970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:33.088270 3970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:33.088369 3970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:38:33.094058 3970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:37439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=32865
--tserver_master_addrs=127.18.125.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:33.095311 3970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:33.096516 3970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:33.107182 3978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:33.107187 3977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:33.107344 3970 server_base.cc:1034] running on GCE node
W20260806 18:38:33.107187 3980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:33.107975 3970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:33.108768 3970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:33.109989 3970 hybrid_clock.cc:648] HybridClock initialized: now 1786041513109972 us; error 29 us; skew 500 ppm
Aug 06 18:38:33 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041513, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:33.112962 3970 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:38:33.113850 3970 webserver.cc:460] Webserver started at http://127.18.125.130:32865/ using document root <none> and password file <none>
I20260806 18:38:33.114434 3970 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:33.114498 3970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:33.118849 3970 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:38:33.121285 3988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:33.122761 3970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:33.122900 3970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "6df4b687354d41049a4310cc028718cd"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "5fec30685d9812b520f33558464f2ac7"
server_key_iv: "84fac2e17495733576667738a4352fc0"
server_key_version: "encryptionkey@0"
I20260806 18:38:33.123487 3970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:33.137552 3970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:33.142119 3970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:33.142377 3970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:33.143254 3970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:33.144996 3995 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:38:33.150475 3970 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:38:33.150570 3970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260806 18:38:33.150651 3970 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:38:33.153340 3970 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:38:33.153419 3970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.001s
I20260806 18:38:33.153781 3995 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Bootstrap starting.
I20260806 18:38:33.161760 3995 log.cc:824] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:33.165900 3970 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:37439
I20260806 18:38:33.166064 4104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:37439 every 8 connection(s)
I20260806 18:38:33.166846 3970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:33.169379 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 3970
I20260806 18:38:33.169878 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:33339
--local_ip_for_outbound_sockets=127.18.125.131
--tserver_master_addrs=127.18.125.190:39117
--webserver_port=37163
--webserver_interface=127.18.125.131
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:38:33.170706 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:33.168877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:44213)
0806 18:38:33.169416 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:33.169631 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":145,"threads_started":1}
W20260806 18:38:33.171679 4105 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:39117 (0 consecutive failures): Network error: Failed to ping master at 127.18.125.190:39117: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
I20260806 18:38:33.172501 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:33.172198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:41919)
0806 18:38:33.172333 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:33.172422 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 18:38:33.172953 3995 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:38:33.173360 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:33.173089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:53163)
0806 18:38:33.173222 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:33.173287 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 18:38:33.173460 3995 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Bootstrap complete.
W20260806 18:38:33.173689 4105 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:38:33.173887 3995 ts_tablet_manager.cc:1397] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20260806 18:38:33.177387 3995 raft_consensus.cc:348] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:33.178054 3995 raft_consensus.cc:729] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df4b687354d41049a4310cc028718cd, State: Initialized, Role: FOLLOWER
I20260806 18:38:33.178666 3995 consensus_queue.cc:260] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:33.179946 3995 ts_tablet_manager.cc:1428] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:38:33.180269 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:33.179818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:60169)
0806 18:38:33.180014 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:33.180075 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260806 18:38:33.181073 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:33.180823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:48051)
0806 18:38:33.180952 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:33.181002 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 18:38:33.300668 4108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:33.300992 4108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:33.301043 4108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:33.305938 4108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:33.306021 4108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:33.306087 4108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:38:33.311928 4108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:33339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=37163
--tserver_master_addrs=127.18.125.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:33.313166 4108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:33.314215 4108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:33.324936 4117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:33.324918 4114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:33.324993 4108 server_base.cc:1034] running on GCE node
W20260806 18:38:33.324918 4115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:33.325768 4108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:33.326520 4108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:33.327817 4108 hybrid_clock.cc:648] HybridClock initialized: now 1786041513327781 us; error 61 us; skew 500 ppm
Aug 06 18:38:33 dist-test-slave-8w9v krb5kdc[2634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041513, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:33.331051 4108 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:38:33.332388 4108 webserver.cc:460] Webserver started at http://127.18.125.131:37163/ using document root <none> and password file <none>
I20260806 18:38:33.333022 4108 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:33.333093 4108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:33.337313 4108 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:38:33.339789 4125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:33.341274 4108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:33.341460 4108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "9879ac85d2854485bee53d9a65de24e4"
format_stamp: "Formatted at 2026-08-06 18:37:50 on dist-test-slave-8w9v"
server_key: "60b5d89fec6a3ca050c2e05fd3954fef"
server_key_iv: "17f1e55c23cc626e186d74a1aa3c5948"
server_key_version: "encryptionkey@0"
I20260806 18:38:33.342043 4108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:33.365439 4108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:33.368520 4108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:33.368741 4108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:33.369405 4108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:33.370273 4108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:33.370321 4108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:33.370393 4108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:33.370433 4108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:33.379992 4108 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:33339
I20260806 18:38:33.380017 4240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:33339 every 8 connection(s)
I20260806 18:38:33.381045 4108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:33.385150 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:33.383258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:42235)
0806 18:38:33.383781 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:33.383903 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":125,"threads_started":1}
W20260806 18:38:33.385725 4241 heartbeater.cc:644] Failed to heartbeat to 127.18.125.190:39117 (0 consecutive failures): Network error: Failed to ping master at 127.18.125.190:39117: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
I20260806 18:38:33.386384 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:33.386128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:42293)
0806 18:38:33.386270 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:33.386324 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 18:38:33.387060 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:33.386841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:37001)
0806 18:38:33.386953 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:33.387008 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 18:38:33.387238 4241 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:38:33.391094 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 4108
I20260806 18:38:34.055295 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.054756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:43383)
0806 18:38:34.055091 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:34.055157 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":88,"threads_started":1}
I20260806 18:38:34.182438 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.181922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:40637)
0806 18:38:34.182258 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:34.182327 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":89,"threads_started":1}
I20260806 18:38:34.388473 4247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.387868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:44569)
0806 18:38:34.388266 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:34.388352 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":91,"threads_started":1}
I20260806 18:38:34.476140 4248 raft_consensus.cc:482] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:34.476403 4248 raft_consensus.cc:504] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:34.477684 4248 leader_election.cc:288] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339), 06d3d04d41944752bae1c881c3a2d526 (127.18.125.129:34633)
Aug 06 18:38:34 dist-test-slave-8w9v krb5kdc[2634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041513, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.131@KRBTEST.COM
Aug 06 18:38:34 dist-test-slave-8w9v krb5kdc[2634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041513, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.129@KRBTEST.COM
I20260806 18:38:34.492091 4249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.478188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:40367 (local address 127.18.125.131:33339)
0806 18:38:34.478568 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:34.478592 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:34.479116 (+ 524us) server_negotiation.cc:407] Connection header received
0806 18:38:34.480023 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:34.480031 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:34.480449 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:34.480760 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:34.481296 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.482351 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.483446 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.483910 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.484093 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:34.487188 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:34.487194 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:34.487207 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:34.487236 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:34.489214 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:34.489897 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:34.489907 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:34.489911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:34.490036 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:34.490422 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:34.490427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:34.490429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:34.490861 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:34.491011 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:34.491582 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 18:38:34.491870 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":136,"threads_started":1}
I20260806 18:38:34.492079 4250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.478184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:38715 (local address 127.18.125.129:34633)
0806 18:38:34.478648 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:34.478653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:34.478681 (+ 28us) server_negotiation.cc:407] Connection header received
0806 18:38:34.479026 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:34.479037 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:34.479315 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:34.479461 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:34.481067 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.482112 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.483446 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.483917 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.484110 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:34.487328 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:34.487334 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:34.487347 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:34.487377 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:34.489322 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:34.489900 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:34.489905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:34.489909 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:34.490038 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:34.490452 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:34.490456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:34.490458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:34.490864 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:34.491010 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:34.491595 (+ 585us) server_negotiation.cc:299] Negotiation successful
0806 18:38:34.491875 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":140,"threads_started":1}
I20260806 18:38:34.492079 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.477992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:34633 (local address 127.18.125.130:38715)
0806 18:38:34.478131 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:34.478175 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:34.478640 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:34.479599 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:34.479609 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:34.480189 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:34.480834 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:34.480869 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.482278 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.482282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:34.483305 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:34.483316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.484082 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.484086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:34.484351 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:34.485102 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:34.485114 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:34.487141 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:34.489458 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:34.489463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:34.489465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:34.489750 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:34.490141 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:34.490144 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:34.490146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:34.490347 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:34.490984 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:34.490990 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:34.491268 (+ 278us) client_negotiation.cc:769] Sending connection context
0806 18:38:34.491470 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 18:38:34.491883 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":175,"spinlock_wait_cycles":21120}
I20260806 18:38:34.492223 4251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.477992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:33339 (local address 127.18.125.130:40367)
0806 18:38:34.478986 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:34.479002 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:34.479125 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:34.480731 (+ 1606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:34.480734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:34.480919 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:34.481101 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:34.481109 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.482497 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.482500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:34.483301 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:34.483310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:34.484051 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:34.484055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:34.484389 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:34.485063 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:34.485095 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:34.486917 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:34.489356 (+ 2439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:34.489363 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:34.489376 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:34.489776 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:34.490141 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:34.490145 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:34.490147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:34.490319 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:34.490983 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:34.490990 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:34.491276 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 18:38:34.491470 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 18:38:34.491975 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"mutex_wait_us":918,"spinlock_wait_cycles":6912,"thread_start_us":112,"threads_started":1}
I20260806 18:38:34.493956 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85b8ecc100942188b324033c75724bc" candidate_uuid: "6df4b687354d41049a4310cc028718cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06d3d04d41944752bae1c881c3a2d526" is_pre_election: true
I20260806 18:38:34.494340 3921 raft_consensus.cc:2454] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6df4b687354d41049a4310cc028718cd in term 1.
W20260806 18:38:34.495503 3989 leader_election.cc:341] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339): Not found: Tablet not found: c85b8ecc100942188b324033c75724bc
I20260806 18:38:34.495790 3990 leader_election.cc:302] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06d3d04d41944752bae1c881c3a2d526, 6df4b687354d41049a4310cc028718cd; no voters: 9879ac85d2854485bee53d9a65de24e4
I20260806 18:38:34.496081 4248 raft_consensus.cc:2781] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 18:38:34.496197 4248 raft_consensus.cc:482] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:34.496255 4248 raft_consensus.cc:3037] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:38:34.498910 4248 raft_consensus.cc:504] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:34.499366 4248 leader_election.cc:288] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [CANDIDATE]: Term 2 election: Requested vote from peers 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339), 06d3d04d41944752bae1c881c3a2d526 (127.18.125.129:34633)
I20260806 18:38:34.499974 3921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85b8ecc100942188b324033c75724bc" candidate_uuid: "6df4b687354d41049a4310cc028718cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "06d3d04d41944752bae1c881c3a2d526"
I20260806 18:38:34.500180 3921 raft_consensus.cc:3037] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 1 FOLLOWER]: Advancing to term 2
W20260806 18:38:34.500393 3989 leader_election.cc:341] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339): Not found: Tablet not found: c85b8ecc100942188b324033c75724bc
I20260806 18:38:34.502866 3921 raft_consensus.cc:2454] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6df4b687354d41049a4310cc028718cd in term 2.
I20260806 18:38:34.503355 3990 leader_election.cc:302] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06d3d04d41944752bae1c881c3a2d526, 6df4b687354d41049a4310cc028718cd; no voters: 9879ac85d2854485bee53d9a65de24e4
I20260806 18:38:34.503599 4248 raft_consensus.cc:2781] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:38:34.503827 4248 raft_consensus.cc:686] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [term 2 LEADER]: Becoming Leader. State: Replica: 6df4b687354d41049a4310cc028718cd, State: Running, Role: LEADER
I20260806 18:38:34.504298 4248 consensus_queue.cc:237] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:34.504819 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.504517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:60203)
0806 18:38:34.504652 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:34.504743 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:38:34.895095 3989 consensus_peers.cc:487] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd -> Peer 9879ac85d2854485bee53d9a65de24e4 (127.18.125.131:33339): Couldn't send request to peer 9879ac85d2854485bee53d9a65de24e4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c85b8ecc100942188b324033c75724bc. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:38:34.923764 3921 raft_consensus.cc:1261] T c85b8ecc100942188b324033c75724bc P 06d3d04d41944752bae1c881c3a2d526 [term 2 FOLLOWER]: Refusing update from remote peer 6df4b687354d41049a4310cc028718cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260806 18:38:34.924700 4248 consensus_queue.cc:1035] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260806 18:38:34.926945 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.926484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:58999)
0806 18:38:34.926735 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:34.926812 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":74,"threads_started":1}
I20260806 18:38:34.929646 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.929348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:52801)
0806 18:38:34.929494 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:34.929555 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 18:38:34.929706 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:34.929447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:35703)
0806 18:38:34.929570 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:34.929626 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 18:38:35.389864 4259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:35.389375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:57881)
0806 18:38:35.389665 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:35.389729 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
I20260806 18:38:35.472518 4260 ts_tablet_manager.cc:927] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Initiating tablet copy from peer 6df4b687354d41049a4310cc028718cd (127.18.125.130:37439)
I20260806 18:38:35.473244 4260 tablet_copy_client.cc:316] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: tablet copy: Beginning tablet copy session from remote peer at address 127.18.125.130:37439
Aug 06 18:38:35 dist-test-slave-8w9v krb5kdc[2634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041513, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.130@KRBTEST.COM
I20260806 18:38:35.485466 4259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:35.473840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:37439 (local address 127.18.125.131:52885)
0806 18:38:35.473999 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:35.474028 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:35.474182 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:35.474817 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:35.474824 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:35.475241 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:35.475359 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:35.475365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:35.476470 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:35.476476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:35.477112 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:35.477121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:35.477767 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:35.477771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:35.477887 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:35.478578 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:35.478591 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:35.480273 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:35.483102 (+ 2829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:35.483117 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:35.483125 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:35.483436 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:35.483849 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:35.483853 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:35.483855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:35.483968 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:35.484644 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:35.484652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:35.484832 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 18:38:35.485013 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 18:38:35.485245 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 18:38:35.485519 4262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:35.474028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:52885 (local address 127.18.125.130:37439)
0806 18:38:35.474379 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:35.474386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:35.474409 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:38:35.474458 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:35.474463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:35.474642 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:35.474812 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:35.475492 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:35.476355 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:35.477249 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:35.477618 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:35.477739 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:35.480439 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:35.480450 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:35.480463 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:35.480513 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:35.482949 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:35.483586 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:35.483594 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:35.483596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:35.483720 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:35.484095 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:35.484099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:35.484101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:35.484492 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:35.484662 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:35.485131 (+ 469us) server_negotiation.cc:299] Negotiation successful
0806 18:38:35.485330 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":79,"threads_started":1}
I20260806 18:38:35.486809 4079 tablet_copy_service.cc:140] P 6df4b687354d41049a4310cc028718cd: Received BeginTabletCopySession request for tablet c85b8ecc100942188b324033c75724bc from peer 9879ac85d2854485bee53d9a65de24e4 ({username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:52885)
I20260806 18:38:35.486992 4079 tablet_copy_service.cc:161] P 6df4b687354d41049a4310cc028718cd: Beginning new tablet copy session on tablet c85b8ecc100942188b324033c75724bc from peer 9879ac85d2854485bee53d9a65de24e4 at {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:52885: session id = 9879ac85d2854485bee53d9a65de24e4-c85b8ecc100942188b324033c75724bc
I20260806 18:38:35.488442 4079 tablet_copy_source_session.cc:205] T c85b8ecc100942188b324033c75724bc P 6df4b687354d41049a4310cc028718cd: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:38:35.493098 4260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85b8ecc100942188b324033c75724bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:35.499912 4260 tablet_copy_client.cc:791] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: tablet copy: Starting download of 0 data blocks...
I20260806 18:38:35.500169 4260 tablet_copy_client.cc:655] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: tablet copy: Starting download of 1 WAL segments...
I20260806 18:38:35.502313 4260 tablet_copy_client.cc:531] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:38:35.504302 4260 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Bootstrap starting.
I20260806 18:38:35.509941 4260 log.cc:824] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:35.516872 4260 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:38:35.517236 4260 tablet_bootstrap.cc:492] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Bootstrap complete.
I20260806 18:38:35.517593 4260 ts_tablet_manager.cc:1397] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Time spent bootstrapping tablet: real 0.014s user 0.014s sys 0.000s
I20260806 18:38:35.520579 4260 raft_consensus.cc:348] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:35.520884 4260 raft_consensus.cc:729] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9879ac85d2854485bee53d9a65de24e4, State: Initialized, Role: FOLLOWER
I20260806 18:38:35.521397 4260 consensus_queue.cc:260] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6df4b687354d41049a4310cc028718cd" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37439 } } peers { permanent_uuid: "9879ac85d2854485bee53d9a65de24e4" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33339 } } peers { permanent_uuid: "06d3d04d41944752bae1c881c3a2d526" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 34633 } }
I20260806 18:38:35.522384 4260 ts_tablet_manager.cc:1428] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:38:35.522745 4259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:35.522454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:51953)
0806 18:38:35.522607 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:35.522668 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 18:38:35.523211 4079 tablet_copy_service.cc:342] P 6df4b687354d41049a4310cc028718cd: Request end of tablet copy session 9879ac85d2854485bee53d9a65de24e4-c85b8ecc100942188b324033c75724bc received from {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:52885
I20260806 18:38:35.523347 4079 tablet_copy_service.cc:433] P 6df4b687354d41049a4310cc028718cd: ending tablet copy session 9879ac85d2854485bee53d9a65de24e4-c85b8ecc100942188b324033c75724bc on tablet c85b8ecc100942188b324033c75724bc with peer 9879ac85d2854485bee53d9a65de24e4
I20260806 18:38:35.523475 4259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:35.523206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:41019)
0806 18:38:35.523340 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:35.523400 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 18:38:35.899331 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:35.898888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:51067)
0806 18:38:35.899131 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:35.899193 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":81,"threads_started":1}
I20260806 18:38:35.899477 4194 raft_consensus.cc:1203] T c85b8ecc100942188b324033c75724bc P 9879ac85d2854485bee53d9a65de24e4 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260806 18:38:35.900683 4259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:35.900338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:50659)
0806 18:38:35.900521 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:35.900573 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 18:38:35.930908 4267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:35.930481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:54113)
0806 18:38:35.930744 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:35.930807 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":73,"threads_started":1}
I20260806 18:38:36.900874 4268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:36.900360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:55809)
0806 18:38:36.900700 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:36.900768 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":120,"threads_started":1}
I20260806 18:38:36.901968 4269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:36.901478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:45103)
0806 18:38:36.901822 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:36.901878 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":94,"threads_started":1}
I20260806 18:38:36.932345 4270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:36.931778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:35157)
0806 18:38:36.932097 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:36.932229 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":83,"threads_started":1}
I20260806 18:38:37.902319 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:37.901748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:60843)
0806 18:38:37.902082 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:37.902148 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":106,"threads_started":1}
I20260806 18:38:37.903348 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:37.902639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:42601)
0806 18:38:37.903145 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:37.903214 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":74,"threads_started":1}
I20260806 18:38:37.933715 4273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:37.933236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:56257)
0806 18:38:37.933560 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:37.933617 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":119,"threads_started":1}
I20260806 18:38:38.904527 4274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:38.903894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:41635)
0806 18:38:38.904293 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:38.904399 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":128,"threads_started":1}
I20260806 18:38:38.904752 4275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:38.904326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:35897)
0806 18:38:38.904564 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:38.904645 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":81,"threads_started":1}
I20260806 18:38:38.905323 4274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:38.905097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:43877)
0806 18:38:38.905215 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:38.905268 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 18:38:38.935334 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:38.934778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:43979)
0806 18:38:38.935157 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:38.935225 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":98,"threads_started":1}
I20260806 18:38:39.906389 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:39.905851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:42991)
0806 18:38:39.906199 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:39.906267 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":138,"threads_started":1}
I20260806 18:38:39.906778 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:39.906343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:58503)
0806 18:38:39.906638 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:39.906696 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":121,"threads_started":1}
I20260806 18:38:39.936761 4280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:39.936303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:40059)
0806 18:38:39.936598 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:39.936660 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":112,"threads_started":1}
I20260806 18:38:40.908010 4281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:40.907521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:34085)
0806 18:38:40.907818 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:40.907899 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":87,"threads_started":1}
I20260806 18:38:40.908039 4282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:40.907494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:43025)
0806 18:38:40.907889 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:40.907946 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":101,"threads_started":1}
I20260806 18:38:40.938028 4283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:40.937574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:42155)
0806 18:38:40.937865 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:40.937927 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":110,"threads_started":1}
I20260806 18:38:41.909400 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:41.908928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:47789)
0806 18:38:41.909234 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:41.909298 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":116,"threads_started":1}
I20260806 18:38:41.909421 4286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:41.908940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:50927)
0806 18:38:41.909281 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:41.909334 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":109,"threads_started":1}
I20260806 18:38:41.939754 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:41.939139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:58099)
0806 18:38:41.939564 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:41.939642 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":150,"threads_started":1}
I20260806 18:38:42.910876 4289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:42.910400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:55897)
0806 18:38:42.910707 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:42.910778 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":112,"threads_started":1}
I20260806 18:38:42.910872 4288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:42.910395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:56131)
0806 18:38:42.910706 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:42.910772 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":119,"threads_started":1}
I20260806 18:38:42.941216 4290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:42.940721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:39953)
0806 18:38:42.941045 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:42.941112 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":83,"threads_started":1}
I20260806 18:38:43.912554 4291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:43.912015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:34109)
0806 18:38:43.912375 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:43.912452 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":172,"threads_started":1}
I20260806 18:38:43.912559 4292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:43.912076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:40459)
0806 18:38:43.912404 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:43.912473 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":166,"threads_started":1}
I20260806 18:38:43.943168 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:43.942632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:32867)
0806 18:38:43.942969 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:43.943038 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":95,"threads_started":1}
I20260806 18:38:44.913990 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:44.913526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:60507)
0806 18:38:44.913819 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:44.913879 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":107,"threads_started":1}
I20260806 18:38:44.914011 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:44.913540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:54165)
0806 18:38:44.913863 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:44.913922 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":126,"threads_started":1}
I20260806 18:38:44.944440 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:44.943962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:51797)
0806 18:38:44.944275 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:44.944342 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":77,"threads_started":1}
I20260806 18:38:45.915313 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:45.914817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:36109)
0806 18:38:45.915122 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:45.915200 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":80,"threads_started":1}
I20260806 18:38:45.915337 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:45.914870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:54659)
0806 18:38:45.915178 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:45.915242 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":111,"threads_started":1}
I20260806 18:38:45.945930 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:45.945444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:50509)
0806 18:38:45.945755 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:45.945817 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":82,"threads_started":1}
I20260806 18:38:46.916965 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.916436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:52123)
0806 18:38:46.916785 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.916854 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":97,"threads_started":1}
I20260806 18:38:46.917384 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.916401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:60149)
0806 18:38:46.917225 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.917288 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":132,"threads_started":1}
I20260806 18:38:46.947463 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:46.946900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:45423)
0806 18:38:46.947280 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:46.947366 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":103,"threads_started":1}
I20260806 18:38:47.918646 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:47.918087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.130:51611)
0806 18:38:47.918488 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:47.918547 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":102,"threads_started":1}
I20260806 18:38:47.918656 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:47.918186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.131:53529)
0806 18:38:47.918505 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:47.918564 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":73,"threads_started":1}
I20260806 18:38:47.948768 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:47.948373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:39117 (local address 127.18.125.129:50689)
0806 18:38:47.948625 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:47.948683 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.190:39117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":81,"threads_started":1}
I20260806 18:38:48.394783 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 3793
I20260806 18:38:48.402674 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 3970
I20260806 18:38:48.409615 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 4108
2026-08-06T18:38:48Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59790 ms)
[----------] 30 tests from SecurityITest (259348 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:48 dist-test-slave-8w9v krb5kdc[4309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:48 dist-test-slave-8w9v krb5kdc[4309](info): set up 2 sockets
Aug 06 18:38:48 dist-test-slave-8w9v krb5kdc[4309](info): commencing operation
krb5kdc: starting...
W20260806 18:38:49.503245 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:38:49 dist-test-slave-8w9v krb5kdc[4309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:38:49.644577 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:35951
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:35951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:49.766607 4322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:49.766927 4322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:49.767033 4322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:49.771184 4322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:49.771277 4322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:49.771314 4322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:49.771373 4322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:49.771432 4322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:49.776925 4322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:35951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:35951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:49.778084 4322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:49.779143 4322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:49.788432 4330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:49.788414 4327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:49.788497 4322 server_base.cc:1034] running on GCE node
W20260806 18:38:49.788401 4328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:49.789260 4322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:49.790401 4322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:49.791596 4322 hybrid_clock.cc:648] HybridClock initialized: now 1786041529791581 us; error 36 us; skew 500 ppm
Aug 06 18:38:49 dist-test-slave-8w9v krb5kdc[4309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041529, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:49.794409 4322 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:38:49.795284 4322 webserver.cc:460] Webserver started at http://127.18.125.190:40649/ using document root <none> and password file <none>
I20260806 18:38:49.795883 4322 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:49.795992 4322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:49.796291 4322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:49.798418 4322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "b4becf269355400ba7b84f6dd924a5b6"
format_stamp: "Formatted at 2026-08-06 18:38:49 on dist-test-slave-8w9v"
server_key: "001e72c0f08dae0d9ec187f75d615382"
server_key_iv: "480ba49388dd19ae59a34b7c98c6ceb1"
server_key_version: "encryptionkey@0"
I20260806 18:38:49.799013 4322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "b4becf269355400ba7b84f6dd924a5b6"
format_stamp: "Formatted at 2026-08-06 18:38:49 on dist-test-slave-8w9v"
server_key: "001e72c0f08dae0d9ec187f75d615382"
server_key_iv: "480ba49388dd19ae59a34b7c98c6ceb1"
server_key_version: "encryptionkey@0"
I20260806 18:38:49.802906 4322 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 18:38:49.805430 4337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:49.806644 4322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:38:49.806794 4322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "b4becf269355400ba7b84f6dd924a5b6"
format_stamp: "Formatted at 2026-08-06 18:38:49 on dist-test-slave-8w9v"
server_key: "001e72c0f08dae0d9ec187f75d615382"
server_key_iv: "480ba49388dd19ae59a34b7c98c6ceb1"
server_key_version: "encryptionkey@0"
I20260806 18:38:49.806963 4322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:49.830266 4322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:49.835542 4322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:49.835788 4322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:49.845052 4322 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:35951
I20260806 18:38:49.845113 4390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:35951 every 8 connection(s)
I20260806 18:38:49.846421 4322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:49.849666 4392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:49.852411 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 4322
I20260806 18:38:49.852562 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:38:49.852892 18934 external_mini_cluster.cc:1428] Setting key 2a3458eadaa78427b4ebaddd774b79a8
I20260806 18:38:49.856766 4392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6: Bootstrap starting.
I20260806 18:38:49.859206 4392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:49.860033 4392 log.cc:824] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:49.862735 4392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6: No bootstrap required, opened a new log
Aug 06 18:38:49 dist-test-slave-8w9v krb5kdc[4309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:49.867504 4392 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4becf269355400ba7b84f6dd924a5b6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35951 } }
I20260806 18:38:49.867853 4392 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:49.867910 4392 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4becf269355400ba7b84f6dd924a5b6, State: Initialized, Role: FOLLOWER
I20260806 18:38:49.868515 4392 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4becf269355400ba7b84f6dd924a5b6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35951 } }
I20260806 18:38:49.868722 4392 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:49.868830 4392 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:49.868939 4392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:49.870119 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:49.854520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59696 (local address 127.18.125.190:35951)
0806 18:38:49.855026 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:49.855043 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:49.855081 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:38:49.855763 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:49.855780 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:49.856347 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:49.856717 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:49.857769 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.858924 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.859915 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:49.860557 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:49.860780 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:49.863730 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:49.863736 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:49.863754 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:49.863790 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:49.866593 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.867105 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.867111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.867117 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.867269 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:49.867564 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:49.867568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:49.867570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:49.868174 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:49.868388 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:49.868709 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 18:38:49.869072 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":156,"threads_started":1}
I20260806 18:38:49.870255 4392 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4becf269355400ba7b84f6dd924a5b6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35951 } }
I20260806 18:38:49.870641 4392 leader_election.cc:302] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4becf269355400ba7b84f6dd924a5b6; no voters:
I20260806 18:38:49.871028 4392 leader_election.cc:288] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:49.871165 4397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:49.872300 4392 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:49.872347 4397 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [term 1 LEADER]: Becoming Leader. State: Replica: b4becf269355400ba7b84f6dd924a5b6, State: Running, Role: LEADER
I20260806 18:38:49.872733 4397 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4becf269355400ba7b84f6dd924a5b6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35951 } }
I20260806 18:38:49.874704 4399 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4becf269355400ba7b84f6dd924a5b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4becf269355400ba7b84f6dd924a5b6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35951 } } }
I20260806 18:38:49.874760 4398 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4becf269355400ba7b84f6dd924a5b6. Latest consensus state: current_term: 1 leader_uuid: "b4becf269355400ba7b84f6dd924a5b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4becf269355400ba7b84f6dd924a5b6" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35951 } } }
I20260806 18:38:49.874943 4399 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:49.874984 4398 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:49.875831 4406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:49.878777 4406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:49.885905 4406 catalog_manager.cc:1347] Generated new cluster ID: 8d9255f7aadd4d5aaa7035d5f762f7d5
I20260806 18:38:49.886008 4406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:49.892040 4406 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:49.893074 4406 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:49.903287 4406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6: Generated new TSK 0
I20260806 18:38:49.904183 4406 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:38:49.940793 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35951
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:50.063772 4418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:50.064067 4418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:50.064112 4418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:50.068373 4418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:50.068439 4418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:50.068501 4418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:38:50.074347 4418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:50.075636 4418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:50.076782 4418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:50.086790 4426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:50.086786 4423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:50.086786 4424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:50.087210 4418 server_base.cc:1034] running on GCE node
I20260806 18:38:50.087638 4418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:50.088428 4418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:50.089603 4418 hybrid_clock.cc:648] HybridClock initialized: now 1786041530089590 us; error 26 us; skew 500 ppm
Aug 06 18:38:50 dist-test-slave-8w9v krb5kdc[4309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041530, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:50.092542 4418 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:38:50.093478 4418 webserver.cc:460] Webserver started at http://127.18.125.129:46823/ using document root <none> and password file <none>
I20260806 18:38:50.094097 4418 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:50.094197 4418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:50.094478 4418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:50.096767 4418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "bffde910f04a4b1c823b587fadd2e94d"
format_stamp: "Formatted at 2026-08-06 18:38:50 on dist-test-slave-8w9v"
server_key: "670c588d93e9d47dad5b586662148fc9"
server_key_iv: "d32a84d81d981745f2649371c6459b60"
server_key_version: "encryptionkey@0"
I20260806 18:38:50.097394 4418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "bffde910f04a4b1c823b587fadd2e94d"
format_stamp: "Formatted at 2026-08-06 18:38:50 on dist-test-slave-8w9v"
server_key: "670c588d93e9d47dad5b586662148fc9"
server_key_iv: "d32a84d81d981745f2649371c6459b60"
server_key_version: "encryptionkey@0"
I20260806 18:38:50.101295 4418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 18:38:50.103925 4433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:50.105330 4418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:38:50.105509 4418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "bffde910f04a4b1c823b587fadd2e94d"
format_stamp: "Formatted at 2026-08-06 18:38:50 on dist-test-slave-8w9v"
server_key: "670c588d93e9d47dad5b586662148fc9"
server_key_iv: "d32a84d81d981745f2649371c6459b60"
server_key_version: "encryptionkey@0"
I20260806 18:38:50.105640 4418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:50.134187 4418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:50.137501 4418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:50.137727 4418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:50.138375 4418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:50.139523 4418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:50.139616 4418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:50.139730 4418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:50.139776 4418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:50.149232 4418 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:36653
I20260806 18:38:50.149250 4548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:36653 every 8 connection(s)
I20260806 18:38:50.150234 4418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:50.158974 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 4418
I20260806 18:38:50.159127 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:50.159457 18934 external_mini_cluster.cc:1428] Setting key 4d2672a7b9c3fe578771724c483ea5e3
Aug 06 18:38:50 dist-test-slave-8w9v krb5kdc[4309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041530, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:50.165975 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.152347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:52205 (local address 127.18.125.190:35951)
0806 18:38:50.152521 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.152525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.153450 (+ 925us) server_negotiation.cc:407] Connection header received
0806 18:38:50.154876 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.154879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.154942 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.155052 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:50.156366 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.156920 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.158115 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.158362 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.158478 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.161402 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:50.161406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:50.161408 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:50.161437 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:50.163249 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:50.164002 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:50.164005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:50.164007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:50.164090 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:50.164584 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:50.164586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:50.164587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:50.164795 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:50.164931 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.165519 (+ 588us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.165694 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 18:38:50.166744 4551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.152757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:35951 (local address 127.18.125.129:52205)
0806 18:38:50.153270 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:50.153315 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:50.154629 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:50.155252 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:50.155262 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:50.155626 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:50.156199 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.156216 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.157084 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.157093 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.157981 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.157990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.158525 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.158529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.158722 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.159362 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:50.159379 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:50.161200 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:50.163404 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:50.163414 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:50.163436 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:50.163872 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:50.164246 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:50.164251 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:50.164254 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:50.164469 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:50.164932 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:50.164959 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:50.165242 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 18:38:50.165412 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 18:38:50.165706 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":130,"threads_started":1}
I20260806 18:38:50.168243 4549 heartbeater.cc:347] Connected to a master server at 127.18.125.190:35951
I20260806 18:38:50.168550 4549 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:50.169221 4549 heartbeater.cc:511] Master 127.18.125.190:35951 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:38:50.171294 4355 ts_manager.cc:194] Registered new tserver with Master: bffde910f04a4b1c823b587fadd2e94d (127.18.125.129:36653)
I20260806 18:38:50.172955 4355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:52205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:38:50.192097 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35951
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:50.310467 4554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:50.310755 4554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:50.310838 4554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:50.315181 4554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:50.315261 4554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:50.315325 4554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:38:50.321082 4554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:50.322264 4554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:50.323345 4554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:50.333616 4562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:50.333644 4559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:50.333617 4560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:50.334038 4554 server_base.cc:1034] running on GCE node
I20260806 18:38:50.334610 4554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:50.335341 4554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:50.336510 4554 hybrid_clock.cc:648] HybridClock initialized: now 1786041530336498 us; error 27 us; skew 500 ppm
Aug 06 18:38:50 dist-test-slave-8w9v krb5kdc[4309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041530, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:50.339725 4554 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:38:50.340821 4554 webserver.cc:460] Webserver started at http://127.18.125.130:34725/ using document root <none> and password file <none>
I20260806 18:38:50.341447 4554 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:50.341514 4554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:50.341771 4554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:50.343950 4554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "331717a2212a40a5afa7badcdd1d9871"
format_stamp: "Formatted at 2026-08-06 18:38:50 on dist-test-slave-8w9v"
server_key: "b265db9bdff4129f28013c4be99f3c4b"
server_key_iv: "3f4010c05c960d93160e0a43c618ce1f"
server_key_version: "encryptionkey@0"
I20260806 18:38:50.344622 4554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "331717a2212a40a5afa7badcdd1d9871"
format_stamp: "Formatted at 2026-08-06 18:38:50 on dist-test-slave-8w9v"
server_key: "b265db9bdff4129f28013c4be99f3c4b"
server_key_iv: "3f4010c05c960d93160e0a43c618ce1f"
server_key_version: "encryptionkey@0"
I20260806 18:38:50.348698 4554 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:38:50.351377 4569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:50.352802 4554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:50.352952 4554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "331717a2212a40a5afa7badcdd1d9871"
format_stamp: "Formatted at 2026-08-06 18:38:50 on dist-test-slave-8w9v"
server_key: "b265db9bdff4129f28013c4be99f3c4b"
server_key_iv: "3f4010c05c960d93160e0a43c618ce1f"
server_key_version: "encryptionkey@0"
I20260806 18:38:50.353077 4554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:50.386322 4554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:50.389743 4554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:50.389964 4554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:50.390615 4554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:50.391531 4554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:50.391601 4554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:50.391676 4554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:50.391729 4554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:50.401242 4554 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:45043
I20260806 18:38:50.401329 4684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:45043 every 8 connection(s)
I20260806 18:38:50.402201 4554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:50.409862 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 4554
I20260806 18:38:50.410032 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:50.410419 18934 external_mini_cluster.cc:1428] Setting key 984ff1b1f5de38b5022b1661c3b51661
Aug 06 18:38:50 dist-test-slave-8w9v krb5kdc[4309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041530, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:50.417570 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.404257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:41573 (local address 127.18.125.190:35951)
0806 18:38:50.404401 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.404405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.405438 (+ 1033us) server_negotiation.cc:407] Connection header received
0806 18:38:50.406789 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.406792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.406841 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.406931 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:50.408319 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.408865 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.410258 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.410624 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.410748 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.413705 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:50.413714 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:50.413717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:50.413746 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:50.415224 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:50.415820 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:50.415823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:50.415825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:50.415913 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:50.416387 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:50.416391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:50.416393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:50.416593 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:50.416708 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.417259 (+ 551us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.417399 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:38:50.418467 4687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.404694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:35951 (local address 127.18.125.130:41573)
0806 18:38:50.405272 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:50.405317 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:50.406577 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:50.407141 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:50.407149 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:50.407561 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:50.408079 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.408095 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.409008 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.409011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.410111 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.410124 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.410831 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.410836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.411078 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.411830 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:50.411854 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:50.413515 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:50.415372 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:50.415378 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:50.415396 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:50.415680 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:50.416032 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:50.416035 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:50.416037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:50.416277 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:50.416708 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:50.416716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:50.417004 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 18:38:50.417197 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 18:38:50.417576 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":158,"threads_started":1}
I20260806 18:38:50.419762 4685 heartbeater.cc:347] Connected to a master server at 127.18.125.190:35951
I20260806 18:38:50.420063 4685 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:38:50.420771 4685 heartbeater.cc:511] Master 127.18.125.190:35951 requested a full tablet report, sending...
I20260806 18:38:50.421944 4355 ts_manager.cc:194] Registered new tserver with Master: 331717a2212a40a5afa7badcdd1d9871 (127.18.125.130:45043)
I20260806 18:38:50.422533 4355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:41573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:38:50.441609 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35951
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:50.557992 4690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:50.558270 4690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:50.558316 4690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:50.562569 4690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:50.562651 4690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:50.562716 4690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:38:50.568557 4690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:50.569746 4690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:50.570823 4690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:50.581148 4695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:50.581161 4698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:50.581205 4690 server_base.cc:1034] running on GCE node
W20260806 18:38:50.581174 4696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:50.581921 4690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:50.582594 4690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:50.583791 4690 hybrid_clock.cc:648] HybridClock initialized: now 1786041530583772 us; error 34 us; skew 500 ppm
Aug 06 18:38:50 dist-test-slave-8w9v krb5kdc[4309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041530, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:50.586706 4690 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:38:50.587637 4690 webserver.cc:460] Webserver started at http://127.18.125.131:33951/ using document root <none> and password file <none>
I20260806 18:38:50.588304 4690 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:50.588400 4690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:50.588639 4690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:50.590766 4690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "4ec354530f4043ccb8fbe27b8380555e"
format_stamp: "Formatted at 2026-08-06 18:38:50 on dist-test-slave-8w9v"
server_key: "9bb3eed3b9442b8ed152d8e20bfe62c5"
server_key_iv: "cd19577ed98b76841d33a1977ff3d4aa"
server_key_version: "encryptionkey@0"
I20260806 18:38:50.591357 4690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "4ec354530f4043ccb8fbe27b8380555e"
format_stamp: "Formatted at 2026-08-06 18:38:50 on dist-test-slave-8w9v"
server_key: "9bb3eed3b9442b8ed152d8e20bfe62c5"
server_key_iv: "cd19577ed98b76841d33a1977ff3d4aa"
server_key_version: "encryptionkey@0"
I20260806 18:38:50.595273 4690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:38:50.597898 4706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:50.599221 4690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:50.599377 4690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "4ec354530f4043ccb8fbe27b8380555e"
format_stamp: "Formatted at 2026-08-06 18:38:50 on dist-test-slave-8w9v"
server_key: "9bb3eed3b9442b8ed152d8e20bfe62c5"
server_key_iv: "cd19577ed98b76841d33a1977ff3d4aa"
server_key_version: "encryptionkey@0"
I20260806 18:38:50.599509 4690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:50.627468 4690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:50.631234 4690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:50.631503 4690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:50.632251 4690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:50.633199 4690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:50.633275 4690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:50.633350 4690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:50.633394 4690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:50.643244 4690 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:41375
I20260806 18:38:50.643270 4821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:41375 every 8 connection(s)
I20260806 18:38:50.644291 4690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:50.649216 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 4690
I20260806 18:38:50.649349 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:50.649727 18934 external_mini_cluster.cc:1428] Setting key b199c4f9936e01a4fb78f2c821d448ef
Aug 06 18:38:50 dist-test-slave-8w9v krb5kdc[4309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041530, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:50.659989 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.646346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:59323 (local address 127.18.125.190:35951)
0806 18:38:50.646534 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.646539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.647463 (+ 924us) server_negotiation.cc:407] Connection header received
0806 18:38:50.648918 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.648921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.648988 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.649069 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:50.650794 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.651566 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.652969 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.653220 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.653342 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.656073 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:50.656077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:50.656079 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:50.656107 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:50.657969 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:50.658514 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:50.658516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:50.658518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:50.658596 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:50.658972 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:50.658974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:50.658975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:50.659164 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:50.659250 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.659750 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.659857 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:38:50.661006 4824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.646803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:35951 (local address 127.18.125.131:59323)
0806 18:38:50.647290 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:50.647335 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:50.648695 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:50.649277 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:50.649288 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:50.649822 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:50.650561 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.650581 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.651706 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.651709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.652825 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.652837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.653392 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.653397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.653599 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.654165 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:50.654183 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:50.655812 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:50.658097 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:50.658103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:50.658118 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:50.658420 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:50.658701 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:50.658704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:50.658705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:50.658863 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:50.659260 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:50.659265 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:50.659533 (+ 268us) client_negotiation.cc:769] Sending connection context
0806 18:38:50.659702 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 18:38:50.659999 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":137,"threads_started":1}
I20260806 18:38:50.662268 4822 heartbeater.cc:347] Connected to a master server at 127.18.125.190:35951
I20260806 18:38:50.662539 4822 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:50.663077 4822 heartbeater.cc:511] Master 127.18.125.190:35951 requested a full tablet report, sending...
I20260806 18:38:50.664083 4355 ts_manager.cc:194] Registered new tserver with Master: 4ec354530f4043ccb8fbe27b8380555e (127.18.125.131:41375)
I20260806 18:38:50.664728 4355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:59323
I20260806 18:38:50.675990 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:50.686213 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.678192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59704 (local address 127.18.125.190:35951)
0806 18:38:50.678370 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.678375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.678539 (+ 164us) server_negotiation.cc:407] Connection header received
0806 18:38:50.678703 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.678706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.678760 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.678849 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:50.679606 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.680202 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.681078 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.681312 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.681435 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.682871 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:50.682875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:50.682878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:50.682907 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:50.684457 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:50.684895 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:50.684898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:50.684900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:50.684985 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:50.685274 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:50.685278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:50.685279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:50.685525 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:50.685674 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.685916 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.686037 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 18:38:50.689296 4355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:50.692579 4355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:50.707587 4834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.701283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60002 (local address 127.18.125.131:41375)
0806 18:38:50.701778 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.701792 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.701817 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:38:50.701901 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.701907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.702140 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.702725 (+ 585us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:50.702877 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.704090 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.706118 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.706840 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.706994 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.707143 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.707322 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.707427 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":155,"threads_started":1}
I20260806 18:38:50.707585 4833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.701084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:41375 (local address 127.0.0.1:60002)
0806 18:38:50.701505 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:50.701556 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:50.701700 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:50.702326 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:50.702332 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:50.702359 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:50.702513 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.702520 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.704268 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.704271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.705982 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.705989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.706982 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.706986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.707100 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.707144 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 18:38:50.707287 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 18:38:50.707382 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":113,"thread_start_us":130,"threads_started":1}
I20260806 18:38:50.707608 4835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.701345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:36653 (local address 127.0.0.1:46782)
0806 18:38:50.701946 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:50.701960 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:50.702035 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:50.703137 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:50.703140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:50.703157 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:50.703268 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.703278 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.704430 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.704433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.705977 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.705989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.707040 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.707044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.707147 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.707176 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:38:50.707238 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:38:50.707383 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":154,"thread_start_us":118,"threads_started":1}
I20260806 18:38:50.708022 4836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.701890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:45043 (local address 127.0.0.1:48186)
0806 18:38:50.702202 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:50.702215 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:50.702305 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:50.703620 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:50.703623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:50.703642 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:50.703728 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.703733 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.705224 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.705228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.706328 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.706340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.707620 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.707624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.707721 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.707744 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:38:50.707822 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 18:38:50.707908 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260806 18:38:50.708428 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.702011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46782 (local address 127.18.125.129:36653)
0806 18:38:50.702714 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.702721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.702747 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:38:50.702827 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.702832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.703008 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.703159 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:50.703399 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.704303 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.706134 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.706886 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.707954 (+ 1068us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.708088 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.708186 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.708287 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":114,"threads_started":1}
I20260806 18:38:50.711100 4482 tablet_service.cc:1467] Processing CreateTablet for tablet a4c6ee68a14f4c6f82b910870080a433 (DEFAULT_TABLE table=test-table [id=5c378430e1d644a092ea06d3c71b7a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:50.711100 4754 tablet_service.cc:1467] Processing CreateTablet for tablet a4c6ee68a14f4c6f82b910870080a433 (DEFAULT_TABLE table=test-table [id=5c378430e1d644a092ea06d3c71b7a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:50.712249 4754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c6ee68a14f4c6f82b910870080a433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:50.712311 4482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c6ee68a14f4c6f82b910870080a433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:50.716584 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.702494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.18.125.130:45043)
0806 18:38:50.703001 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.703023 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.703049 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:38:50.703126 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.703131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.703295 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.703433 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:50.703853 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.704883 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.706475 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.707299 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.707431 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.707555 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.716230 (+ 8675us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.716407 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":136,"threads_started":1}
I20260806 18:38:50.719077 4839 tablet_bootstrap.cc:492] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e: Bootstrap starting.
I20260806 18:38:50.720228 4840 tablet_bootstrap.cc:492] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d: Bootstrap starting.
I20260806 18:38:50.719818 4619 tablet_service.cc:1467] Processing CreateTablet for tablet a4c6ee68a14f4c6f82b910870080a433 (DEFAULT_TABLE table=test-table [id=5c378430e1d644a092ea06d3c71b7a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:50.721058 4619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c6ee68a14f4c6f82b910870080a433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:50.721565 4839 tablet_bootstrap.cc:654] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:50.722378 4840 tablet_bootstrap.cc:654] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:50.722635 4839 log.cc:824] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:50.723215 4840 log.cc:824] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:50.725153 4839 tablet_bootstrap.cc:492] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e: No bootstrap required, opened a new log
I20260806 18:38:50.725378 4839 ts_tablet_manager.cc:1397] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:38:50.725479 4840 tablet_bootstrap.cc:492] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d: No bootstrap required, opened a new log
I20260806 18:38:50.725660 4840 ts_tablet_manager.cc:1397] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:38:50.728211 4843 tablet_bootstrap.cc:492] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871: Bootstrap starting.
I20260806 18:38:50.729770 4839 raft_consensus.cc:348] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec354530f4043ccb8fbe27b8380555e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41375 } } peers { permanent_uuid: "331717a2212a40a5afa7badcdd1d9871" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45043 } } peers { permanent_uuid: "bffde910f04a4b1c823b587fadd2e94d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 36653 } }
I20260806 18:38:50.730096 4839 raft_consensus.cc:374] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:50.730225 4839 raft_consensus.cc:729] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec354530f4043ccb8fbe27b8380555e, State: Initialized, Role: FOLLOWER
I20260806 18:38:50.730365 4840 raft_consensus.cc:348] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec354530f4043ccb8fbe27b8380555e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41375 } } peers { permanent_uuid: "331717a2212a40a5afa7badcdd1d9871" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45043 } } peers { permanent_uuid: "bffde910f04a4b1c823b587fadd2e94d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 36653 } }
I20260806 18:38:50.730674 4840 raft_consensus.cc:374] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:50.730743 4840 raft_consensus.cc:729] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bffde910f04a4b1c823b587fadd2e94d, State: Initialized, Role: FOLLOWER
I20260806 18:38:50.730782 4839 consensus_queue.cc:260] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec354530f4043ccb8fbe27b8380555e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41375 } } peers { permanent_uuid: "331717a2212a40a5afa7badcdd1d9871" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45043 } } peers { permanent_uuid: "bffde910f04a4b1c823b587fadd2e94d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 36653 } }
I20260806 18:38:50.731005 4843 tablet_bootstrap.cc:654] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:50.731208 4840 consensus_queue.cc:260] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec354530f4043ccb8fbe27b8380555e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41375 } } peers { permanent_uuid: "331717a2212a40a5afa7badcdd1d9871" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45043 } } peers { permanent_uuid: "bffde910f04a4b1c823b587fadd2e94d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 36653 } }
I20260806 18:38:50.731799 4822 heartbeater.cc:503] Master 127.18.125.190:35951 was elected leader, sending a full tablet report...
I20260806 18:38:50.732337 4840 ts_tablet_manager.cc:1428] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 18:38:50.732566 4843 log.cc:824] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:50.732717 4549 heartbeater.cc:503] Master 127.18.125.190:35951 was elected leader, sending a full tablet report...
I20260806 18:38:50.733244 4839 ts_tablet_manager.cc:1428] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 18:38:50.735069 4843 tablet_bootstrap.cc:492] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871: No bootstrap required, opened a new log
I20260806 18:38:50.735292 4843 ts_tablet_manager.cc:1397] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:38:50.738739 4843 raft_consensus.cc:348] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec354530f4043ccb8fbe27b8380555e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41375 } } peers { permanent_uuid: "331717a2212a40a5afa7badcdd1d9871" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45043 } } peers { permanent_uuid: "bffde910f04a4b1c823b587fadd2e94d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 36653 } }
I20260806 18:38:50.739023 4843 raft_consensus.cc:374] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:50.739099 4843 raft_consensus.cc:729] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 331717a2212a40a5afa7badcdd1d9871, State: Initialized, Role: FOLLOWER
I20260806 18:38:50.739583 4843 consensus_queue.cc:260] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec354530f4043ccb8fbe27b8380555e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41375 } } peers { permanent_uuid: "331717a2212a40a5afa7badcdd1d9871" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45043 } } peers { permanent_uuid: "bffde910f04a4b1c823b587fadd2e94d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 36653 } }
I20260806 18:38:50.740397 4685 heartbeater.cc:503] Master 127.18.125.190:35951 was elected leader, sending a full tablet report...
I20260806 18:38:50.740715 4843 ts_tablet_manager.cc:1428] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:50.809696 4845 raft_consensus.cc:482] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:50.809991 4845 raft_consensus.cc:504] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec354530f4043ccb8fbe27b8380555e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41375 } } peers { permanent_uuid: "331717a2212a40a5afa7badcdd1d9871" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45043 } } peers { permanent_uuid: "bffde910f04a4b1c823b587fadd2e94d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 36653 } }
I20260806 18:38:50.811332 4845 leader_election.cc:288] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ec354530f4043ccb8fbe27b8380555e (127.18.125.131:41375), 331717a2212a40a5afa7badcdd1d9871 (127.18.125.130:45043)
I20260806 18:38:50.816213 4834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.811780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:54075 (local address 127.18.125.131:41375)
0806 18:38:50.811956 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.811961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.812159 (+ 198us) server_negotiation.cc:407] Connection header received
0806 18:38:50.812340 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.812345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.812416 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.812501 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:50.812970 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.813597 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.815010 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.815504 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.815608 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.815654 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.815927 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.816040 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:38:50.816462 4849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.812178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:45043 (local address 127.18.125.129:51027)
0806 18:38:50.812512 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:50.812526 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:50.812679 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:50.812969 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:50.812972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:50.812989 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:50.813103 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.813109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.814082 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.814085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.815041 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.815052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.815815 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.815818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.815943 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.815979 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 18:38:50.816058 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:38:50.816157 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":1,"thread_start_us":102,"threads_started":1}
I20260806 18:38:50.816632 4848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.811675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:41375 (local address 127.18.125.129:54075)
0806 18:38:50.812014 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:50.812028 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:50.812215 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:50.812594 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:50.812598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:50.812625 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:50.812815 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.812823 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.813817 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.813821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.814762 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:50.814774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.815644 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.815648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:50.815768 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.815797 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:38:50.815872 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:38:50.816263 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"spinlock_wait_cycles":9600,"thread_start_us":159,"threads_started":1}
I20260806 18:38:50.816707 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.812372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:51027 (local address 127.18.125.130:45043)
0806 18:38:50.812515 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.812522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.812598 (+ 76us) server_negotiation.cc:407] Connection header received
0806 18:38:50.812800 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.812804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.812863 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.812943 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:50.813255 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.813911 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.815196 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.815680 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.815809 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.815880 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.816276 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.816452 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:38:50.817431 4775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c6ee68a14f4c6f82b910870080a433" candidate_uuid: "bffde910f04a4b1c823b587fadd2e94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec354530f4043ccb8fbe27b8380555e" is_pre_election: true
I20260806 18:38:50.817453 4639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c6ee68a14f4c6f82b910870080a433" candidate_uuid: "bffde910f04a4b1c823b587fadd2e94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "331717a2212a40a5afa7badcdd1d9871" is_pre_election: true
I20260806 18:38:50.817763 4639 raft_consensus.cc:2454] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bffde910f04a4b1c823b587fadd2e94d in term 0.
I20260806 18:38:50.817770 4775 raft_consensus.cc:2454] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bffde910f04a4b1c823b587fadd2e94d in term 0.
I20260806 18:38:50.818385 4437 leader_election.cc:302] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ec354530f4043ccb8fbe27b8380555e, bffde910f04a4b1c823b587fadd2e94d; no voters:
I20260806 18:38:50.818724 4845 raft_consensus.cc:2781] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:50.818820 4845 raft_consensus.cc:482] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:50.818877 4845 raft_consensus.cc:3037] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:50.820024 4845 raft_consensus.cc:504] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec354530f4043ccb8fbe27b8380555e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41375 } } peers { permanent_uuid: "331717a2212a40a5afa7badcdd1d9871" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45043 } } peers { permanent_uuid: "bffde910f04a4b1c823b587fadd2e94d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 36653 } }
I20260806 18:38:50.820488 4845 leader_election.cc:288] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [CANDIDATE]: Term 1 election: Requested vote from peers 4ec354530f4043ccb8fbe27b8380555e (127.18.125.131:41375), 331717a2212a40a5afa7badcdd1d9871 (127.18.125.130:45043)
I20260806 18:38:50.821334 4775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c6ee68a14f4c6f82b910870080a433" candidate_uuid: "bffde910f04a4b1c823b587fadd2e94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec354530f4043ccb8fbe27b8380555e"
I20260806 18:38:50.821475 4775 raft_consensus.cc:3037] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:50.821349 4639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c6ee68a14f4c6f82b910870080a433" candidate_uuid: "bffde910f04a4b1c823b587fadd2e94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "331717a2212a40a5afa7badcdd1d9871"
I20260806 18:38:50.821734 4639 raft_consensus.cc:3037] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:50.823918 4775 raft_consensus.cc:2454] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bffde910f04a4b1c823b587fadd2e94d in term 1.
I20260806 18:38:50.824628 4639 raft_consensus.cc:2454] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bffde910f04a4b1c823b587fadd2e94d in term 1.
I20260806 18:38:50.824618 4437 leader_election.cc:302] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ec354530f4043ccb8fbe27b8380555e, bffde910f04a4b1c823b587fadd2e94d; no voters:
I20260806 18:38:50.825086 4845 raft_consensus.cc:2781] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:50.825367 4845 raft_consensus.cc:686] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [term 1 LEADER]: Becoming Leader. State: Replica: bffde910f04a4b1c823b587fadd2e94d, State: Running, Role: LEADER
I20260806 18:38:50.825735 4845 consensus_queue.cc:237] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec354530f4043ccb8fbe27b8380555e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41375 } } peers { permanent_uuid: "331717a2212a40a5afa7badcdd1d9871" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45043 } } peers { permanent_uuid: "bffde910f04a4b1c823b587fadd2e94d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 36653 } }
I20260806 18:38:50.829282 4350 catalog_manager.cc:5495] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d reported cstate change: term changed from 0 to 1, leader changed from <none> to bffde910f04a4b1c823b587fadd2e94d (127.18.125.129). New cstate: current_term: 1 leader_uuid: "bffde910f04a4b1c823b587fadd2e94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ec354530f4043ccb8fbe27b8380555e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "331717a2212a40a5afa7badcdd1d9871" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bffde910f04a4b1c823b587fadd2e94d" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 36653 } health_report { overall_health: HEALTHY } } }
I20260806 18:38:50.863054 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.859727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46786 (local address 127.18.125.129:36653)
0806 18:38:50.859864 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:50.859868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:50.859882 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:38:50.859960 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:50.859963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:50.860018 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:50.860107 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:50.860387 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.860894 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.861717 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:50.861935 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:50.862016 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:50.862287 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:50.862712 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:50.862806 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:50.862865 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 18:38:50.862920 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:38:50.869371 4775 raft_consensus.cc:1261] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e [term 1 FOLLOWER]: Refusing update from remote peer bffde910f04a4b1c823b587fadd2e94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:50.869371 4639 raft_consensus.cc:1261] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871 [term 1 FOLLOWER]: Refusing update from remote peer bffde910f04a4b1c823b587fadd2e94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:50.870136 4850 consensus_queue.cc:1035] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ec354530f4043ccb8fbe27b8380555e" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 41375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:50.870360 4845 consensus_queue.cc:1035] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "331717a2212a40a5afa7badcdd1d9871" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:50.878952 4854 mvcc.cc:205] Tried to move back new op lower bound from 7315626110432796672 to 7315626110265905152. Current Snapshot: MvccSnapshot[applied={T|T < 7315626110432796672}]
I20260806 18:38:50.888541 4350 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:50.888752 4350 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:50.891467 4350 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4becf269355400ba7b84f6dd924a5b6: Sending DeleteTablet for 3 replicas of tablet a4c6ee68a14f4c6f82b910870080a433
I20260806 18:38:50.892450 4482 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c6ee68a14f4c6f82b910870080a433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:50 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:46782
I20260806 18:38:50.892481 4754 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c6ee68a14f4c6f82b910870080a433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:50 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:60002
I20260806 18:38:50.892529 4619 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c6ee68a14f4c6f82b910870080a433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:50 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:48186
I20260806 18:38:50.893072 4865 tablet_replica.cc:330] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e: stopping tablet replica
I20260806 18:38:50.893229 4864 tablet_replica.cc:330] T a4c6ee68a14f4c6f82b910870080a433 P bffde910f04a4b1c823b587fadd2e94d: stopping tablet replica
I20260806 18:38:50.893517 4865 raft_consensus.cc:2229] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:50.893741 4863 tablet_replica.cc:330] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871: stopping tablet replica
I20260806 18:38:50.893846 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 4418
I20260806 18:38:50.894095 4863 raft_consensus.cc:2229] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:50.894579 4863 raft_consensus.cc:2258] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:50.896890 4865 raft_consensus.cc:2258] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:50.897238 4863 ts_tablet_manager.cc:1905] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:50.898348 4865 ts_tablet_manager.cc:1905] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:50.901006 4865 ts_tablet_manager.cc:1918] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:50.901144 4865 log.cc:1196] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/a4c6ee68a14f4c6f82b910870080a433
I20260806 18:38:50.901284 4863 ts_tablet_manager.cc:1918] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:50.901449 4863 log.cc:1196] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/a4c6ee68a14f4c6f82b910870080a433
I20260806 18:38:50.901600 4865 ts_tablet_manager.cc:1939] T a4c6ee68a14f4c6f82b910870080a433 P 4ec354530f4043ccb8fbe27b8380555e: Deleting consensus metadata
I20260806 18:38:50.901803 4863 ts_tablet_manager.cc:1939] T a4c6ee68a14f4c6f82b910870080a433 P 331717a2212a40a5afa7badcdd1d9871: Deleting consensus metadata
I20260806 18:38:50.903033 4340 catalog_manager.cc:4841] TS 4ec354530f4043ccb8fbe27b8380555e (127.18.125.131:41375): tablet a4c6ee68a14f4c6f82b910870080a433 (table test-table [id=5c378430e1d644a092ea06d3c71b7a78]) successfully deleted
I20260806 18:38:50.903374 4340 catalog_manager.cc:4841] TS 331717a2212a40a5afa7badcdd1d9871 (127.18.125.130:45043): tablet a4c6ee68a14f4c6f82b910870080a433 (table test-table [id=5c378430e1d644a092ea06d3c71b7a78]) successfully deleted
W20260806 18:38:50.903978 4338 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:36653 (error 108)
I20260806 18:38:50.904276 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 4554
I20260806 18:38:50.905900 4836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:50.905421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:36653 (local address 127.0.0.1:46790)
0806 18:38:50.905645 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:50.905746 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:36653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260806 18:38:50.906140 4338 catalog_manager.cc:4584] TS bffde910f04a4b1c823b587fadd2e94d (127.18.125.129:36653): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a4c6ee68a14f4c6f82b910870080a433: Network error: Client connection negotiation failed: client connection to 127.18.125.129:36653: connect: Connection refused (error 111)
I20260806 18:38:50.913334 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 4690
I20260806 18:38:50.923139 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 4322
2026-08-06T18:38:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2512 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T18:38:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:50Z Disabled control of system clock
I20260806 18:38:50.956773 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:35653
--webserver_interface=127.18.125.190
--webserver_port=0
--builtin_ntp_servers=127.18.125.148:39321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:35653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:51.079720 4870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:51.080015 4870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:51.080053 4870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:51.084363 4870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:51.084435 4870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:51.084451 4870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:51.084470 4870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:51.084514 4870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:51.089489 4870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:39321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:35653
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:35653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:51.090767 4870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:51.092298 4870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:51.101475 4878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:51.101529 4870 server_base.cc:1034] running on GCE node
W20260806 18:38:51.101436 4876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:51.101460 4875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:51.102420 4870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:51.103660 4870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:51.104851 4870 hybrid_clock.cc:648] HybridClock initialized: now 1786041531104821 us; error 47 us; skew 500 ppm
I20260806 18:38:51.106914 4870 webserver.cc:460] Webserver started at http://127.18.125.190:35549/ using document root <none> and password file <none>
I20260806 18:38:51.107553 4870 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:51.107652 4870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:51.107918 4870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:51.110157 4870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "72facebca074461a93cb780a09141024"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "2e12344560ea8f8c80b554b6aeec719e"
server_key_iv: "30ec612c2eaf2de217502796ac0ab377"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.110742 4870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "72facebca074461a93cb780a09141024"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "2e12344560ea8f8c80b554b6aeec719e"
server_key_iv: "30ec612c2eaf2de217502796ac0ab377"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.114737 4870 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:38:51.117535 4884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:51.118806 4870 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:38:51.118976 4870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "72facebca074461a93cb780a09141024"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "2e12344560ea8f8c80b554b6aeec719e"
server_key_iv: "30ec612c2eaf2de217502796ac0ab377"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.119107 4870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:51.150918 4870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:51.151705 4870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:51.151932 4870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:51.159418 4870 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:35653
I20260806 18:38:51.159440 4938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:35653 every 8 connection(s)
I20260806 18:38:51.160447 4870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:51.163765 4939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:51.164000 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 4870
I20260806 18:38:51.164201 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:38:51.164515 18934 external_mini_cluster.cc:1428] Setting key 04381e6f4ac0a5a6aa9f7e9c84c65bb4
I20260806 18:38:51.170737 4939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024: Bootstrap starting.
I20260806 18:38:51.173170 4939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:51.174067 4939 log.cc:824] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:51.174162 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.166004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.18.125.190:35653)
0806 18:38:51.166706 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:51.166722 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:51.166758 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:38:51.167445 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:51.167464 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:51.167862 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:51.168243 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:51.168699 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.169677 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.170626 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.171167 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.171359 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.172353 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:51.172358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:51.172371 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:51.172775 (+ 404us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:51.172836 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:51.172856 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:51.172975 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:51.173091 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 18:38:51.173415 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":150,"threads_started":1}
I20260806 18:38:51.176287 4939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024: No bootstrap required, opened a new log
I20260806 18:38:51.180357 4939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72facebca074461a93cb780a09141024" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35653 } }
I20260806 18:38:51.180601 4939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:51.180644 4939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72facebca074461a93cb780a09141024, State: Initialized, Role: FOLLOWER
I20260806 18:38:51.181023 4939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72facebca074461a93cb780a09141024" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35653 } }
I20260806 18:38:51.181136 4939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:51.181181 4939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:51.181252 4939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:51.182255 4939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72facebca074461a93cb780a09141024" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35653 } }
I20260806 18:38:51.182554 4939 leader_election.cc:302] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72facebca074461a93cb780a09141024; no voters:
I20260806 18:38:51.182824 4939 leader_election.cc:288] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:51.182972 4944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:51.183225 4944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [term 1 LEADER]: Becoming Leader. State: Replica: 72facebca074461a93cb780a09141024, State: Running, Role: LEADER
I20260806 18:38:51.183610 4944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72facebca074461a93cb780a09141024" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35653 } }
I20260806 18:38:51.184185 4939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:51.185534 4946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72facebca074461a93cb780a09141024" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72facebca074461a93cb780a09141024" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35653 } } }
I20260806 18:38:51.185703 4946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:51.185526 4945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72facebca074461a93cb780a09141024. Latest consensus state: current_term: 1 leader_uuid: "72facebca074461a93cb780a09141024" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72facebca074461a93cb780a09141024" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35653 } } }
I20260806 18:38:51.186029 4945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:51.186795 4955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:51.189755 4955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:51.195763 4955 catalog_manager.cc:1347] Generated new cluster ID: 2c76a86d9f124b488fc7f73faf4da35e
I20260806 18:38:51.195863 4955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:51.216177 4955 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:51.217595 4955 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:51.225627 4955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024: Generated new TSK 0
I20260806 18:38:51.226512 4955 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:51.241591 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35653
--builtin_ntp_servers=127.18.125.148:39321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:51.378813 4963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:51.379096 4963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:51.379136 4963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:51.383224 4963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:51.383292 4963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:51.383354 4963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:38:51.388080 4963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:39321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:51.389292 4963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:51.390684 4963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:51.400902 4971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:51.400887 4968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:51.400928 4963 server_base.cc:1034] running on GCE node
W20260806 18:38:51.400885 4969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:51.401646 4963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:51.402356 4963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:51.403553 4963 hybrid_clock.cc:648] HybridClock initialized: now 1786041531403524 us; error 53 us; skew 500 ppm
I20260806 18:38:51.405607 4963 webserver.cc:460] Webserver started at http://127.18.125.129:40503/ using document root <none> and password file <none>
I20260806 18:38:51.406272 4963 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:51.406375 4963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:51.406627 4963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:51.408926 4963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "01b13497a9984bb5b7aad5a52d5981fb"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "88bfa61c5053d58f5cd5c9ad68cb8a71"
server_key_iv: "1f6969dcdf666fa00af3c8f2906982aa"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.409567 4963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "01b13497a9984bb5b7aad5a52d5981fb"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "88bfa61c5053d58f5cd5c9ad68cb8a71"
server_key_iv: "1f6969dcdf666fa00af3c8f2906982aa"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.413661 4963 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 18:38:51.416467 4977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:51.417833 4963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:38:51.417997 4963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "01b13497a9984bb5b7aad5a52d5981fb"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "88bfa61c5053d58f5cd5c9ad68cb8a71"
server_key_iv: "1f6969dcdf666fa00af3c8f2906982aa"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.418134 4963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:51.441813 4963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:51.442606 4963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:51.442833 4963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:51.443503 4963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:51.444622 4963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:51.444695 4963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:51.444767 4963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:51.444819 4963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:51.454710 4963 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:42531
I20260806 18:38:51.454741 5092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:42531 every 8 connection(s)
I20260806 18:38:51.455699 4963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:51.460695 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 4963
I20260806 18:38:51.460826 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:51.461141 18934 external_mini_cluster.cc:1428] Setting key a2958c367a79ffa576ffe38742e1a05b
I20260806 18:38:51.463200 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35653
--builtin_ntp_servers=127.18.125.148:39321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:51.465935 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.457682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:43395 (local address 127.18.125.190:35653)
0806 18:38:51.457853 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:51.457858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:51.458939 (+ 1081us) server_negotiation.cc:407] Connection header received
0806 18:38:51.460461 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:51.460464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:51.460520 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:51.460598 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:51.461884 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.462576 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.463874 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.464110 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.464275 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.465411 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:51.465415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:51.465418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:51.465431 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:51.465460 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:51.465468 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:51.465549 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:51.465710 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 18:38:51.465781 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:38:51.466815 5095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.458135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:35653 (local address 127.18.125.129:43395)
0806 18:38:51.458786 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:51.458827 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:51.460195 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:51.460825 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:51.460837 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:51.460960 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:51.461694 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.461711 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.462712 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.462716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.463737 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.463748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.464318 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.464322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.464499 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.465144 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:51.465146 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:51.465185 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:51.465252 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:51.465552 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:51.465564 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:51.465571 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:38:51.465710 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 18:38:51.466012 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260806 18:38:51.468178 5093 heartbeater.cc:347] Connected to a master server at 127.18.125.190:35653
I20260806 18:38:51.468461 5093 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:51.469034 5093 heartbeater.cc:511] Master 127.18.125.190:35653 requested a full tablet report, sending...
I20260806 18:38:51.470887 4902 ts_manager.cc:194] Registered new tserver with Master: 01b13497a9984bb5b7aad5a52d5981fb (127.18.125.129:42531)
I20260806 18:38:51.472777 4902 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.129:43395
W20260806 18:38:51.583704 5096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:51.584017 5096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:51.584059 5096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:51.588601 5096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:51.588680 5096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:51.588742 5096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:38:51.593808 5096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:39321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:51.594966 5096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:51.596462 5096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:51.607704 5104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:51.607805 5096 server_base.cc:1034] running on GCE node
W20260806 18:38:51.607764 5101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:51.607704 5102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:51.608588 5096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:51.609402 5096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:51.610606 5096 hybrid_clock.cc:648] HybridClock initialized: now 1786041531610576 us; error 55 us; skew 500 ppm
I20260806 18:38:51.612908 5096 webserver.cc:460] Webserver started at http://127.18.125.130:41515/ using document root <none> and password file <none>
I20260806 18:38:51.613566 5096 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:51.613669 5096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:51.613925 5096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:51.616271 5096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "b7d1d94f486e4f57ac5af5b438d84456"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "4ea24522392fb4e5d58d7413baacb47c"
server_key_iv: "9353a4bfab32996ad216b583bf344d99"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.616925 5096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "b7d1d94f486e4f57ac5af5b438d84456"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "4ea24522392fb4e5d58d7413baacb47c"
server_key_iv: "9353a4bfab32996ad216b583bf344d99"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.621081 5096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:38:51.624015 5110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:51.625362 5096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:51.625516 5096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "b7d1d94f486e4f57ac5af5b438d84456"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "4ea24522392fb4e5d58d7413baacb47c"
server_key_iv: "9353a4bfab32996ad216b583bf344d99"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.625641 5096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:51.646569 5096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:51.647380 5096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:51.647600 5096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:51.648345 5096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:51.649385 5096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:51.649436 5096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:51.649505 5096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:51.649538 5096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:51.659497 5096 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:37509
I20260806 18:38:51.659539 5224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:37509 every 8 connection(s)
I20260806 18:38:51.660478 5096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:51.660966 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 5096
I20260806 18:38:51.661083 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:51.661427 18934 external_mini_cluster.cc:1428] Setting key 64886f0813059ecfffa75e3990869e56
I20260806 18:38:51.663762 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35653
--builtin_ntp_servers=127.18.125.148:39321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:51.671866 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.662623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:52237 (local address 127.18.125.190:35653)
0806 18:38:51.662776 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:51.662781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:51.664109 (+ 1328us) server_negotiation.cc:407] Connection header received
0806 18:38:51.665829 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:51.665831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:51.665879 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:51.665964 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:51.667213 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.667732 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.669644 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.669858 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.669954 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.671378 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:51.671382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:51.671384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:51.671397 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:51.671415 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:51.671423 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:51.671478 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:51.671691 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 18:38:51.671745 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:38:51.673246 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.663168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:35653 (local address 127.18.125.130:52237)
0806 18:38:51.663933 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:51.663996 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:51.665572 (+ 1576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:51.666184 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:51.666196 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:51.666321 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:51.667058 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.667080 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.667872 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.667878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.669504 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.669521 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.670042 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.670050 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.670305 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.671024 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:51.671028 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:51.671089 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:51.671189 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:51.671514 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:51.671527 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:51.671535 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 18:38:51.671690 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 18:38:51.672037 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":141,"threads_started":1}
I20260806 18:38:51.674626 5225 heartbeater.cc:347] Connected to a master server at 127.18.125.190:35653
I20260806 18:38:51.674886 5225 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:51.675521 5225 heartbeater.cc:511] Master 127.18.125.190:35653 requested a full tablet report, sending...
I20260806 18:38:51.676738 4902 ts_manager.cc:194] Registered new tserver with Master: b7d1d94f486e4f57ac5af5b438d84456 (127.18.125.130:37509)
I20260806 18:38:51.677333 4902 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.130:52237
W20260806 18:38:51.788457 5228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:51.788761 5228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:51.788803 5228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:51.793164 5228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:51.793239 5228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:51.793300 5228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:38:51.798352 5228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:39321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:35653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:51.799553 5228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:51.801100 5228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:51.810670 5236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:51.810680 5233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:51.810912 5228 server_base.cc:1034] running on GCE node
W20260806 18:38:51.810762 5234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:51.811501 5228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:51.812198 5228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:51.813411 5228 hybrid_clock.cc:648] HybridClock initialized: now 1786041531813378 us; error 27 us; skew 500 ppm
I20260806 18:38:51.815596 5228 webserver.cc:460] Webserver started at http://127.18.125.131:45711/ using document root <none> and password file <none>
I20260806 18:38:51.816360 5228 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:51.816470 5228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:51.816742 5228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:51.819036 5228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "1eebca1b5d1d4671a2f26a37c7087cde"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "7398793b7b7998c6d5c7c80fad3b3784"
server_key_iv: "e9ad7dff983a045b8a47ac3fb9cbec71"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.819715 5228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "1eebca1b5d1d4671a2f26a37c7087cde"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "7398793b7b7998c6d5c7c80fad3b3784"
server_key_iv: "e9ad7dff983a045b8a47ac3fb9cbec71"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.824579 5228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 18:38:51.827343 5242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:51.828676 5228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:51.828826 5228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "1eebca1b5d1d4671a2f26a37c7087cde"
format_stamp: "Formatted at 2026-08-06 18:38:51 on dist-test-slave-8w9v"
server_key: "7398793b7b7998c6d5c7c80fad3b3784"
server_key_iv: "e9ad7dff983a045b8a47ac3fb9cbec71"
server_key_version: "encryptionkey@0"
I20260806 18:38:51.829248 5228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:51.861851 5228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:51.862676 5228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:51.862893 5228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:51.863557 5228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:51.864667 5228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:51.864741 5228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:51.864816 5228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:51.864871 5228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:51.874655 5228 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:33675
I20260806 18:38:51.874720 5357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:33675 every 8 connection(s)
I20260806 18:38:51.875742 5228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:51.882262 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 5228
I20260806 18:38:51.882424 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:51.882776 18934 external_mini_cluster.cc:1428] Setting key 59b253115153b2ecffede22587111dae
I20260806 18:38:51.887856 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.877980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:33493 (local address 127.18.125.190:35653)
0806 18:38:51.878151 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:51.878155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:51.879821 (+ 1666us) server_negotiation.cc:407] Connection header received
0806 18:38:51.881551 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:51.881555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:51.881624 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:51.881743 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:51.883223 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.884029 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.885732 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.885974 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.886098 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.887314 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:51.887323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:51.887325 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:51.887338 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:51.887357 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:51.887365 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:51.887447 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:51.887633 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 18:38:51.887701 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 18:38:51.888963 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.878409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:35653 (local address 127.18.125.131:33493)
0806 18:38:51.879454 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:51.879543 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:51.881259 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:51.881953 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:51.882090 (+ 137us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:51.882217 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:51.883010 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.883031 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.884271 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.884276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.885582 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.885596 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.886150 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.886155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.886351 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.887004 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:51.887006 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:51.887052 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:51.887132 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:51.887464 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:51.887477 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:51.887483 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:38:51.887630 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 18:38:51.887980 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":223,"threads_started":1}
I20260806 18:38:51.890201 5358 heartbeater.cc:347] Connected to a master server at 127.18.125.190:35653
I20260806 18:38:51.890501 5358 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:51.891117 5358 heartbeater.cc:511] Master 127.18.125.190:35653 requested a full tablet report, sending...
I20260806 18:38:51.892354 4902 ts_manager.cc:194] Registered new tserver with Master: 1eebca1b5d1d4671a2f26a37c7087cde (127.18.125.131:33675)
I20260806 18:38:51.892979 4902 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.131:33493
I20260806 18:38:51.897601 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:51.904007 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.899831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.18.125.190:35653)
0806 18:38:51.899984 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:51.899989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:51.900209 (+ 220us) server_negotiation.cc:407] Connection header received
0806 18:38:51.900402 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:51.900405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:51.900450 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:51.900543 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:51.900857 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.901404 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.902295 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.902519 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.902630 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.903526 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:51.903529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:51.903532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:51.903545 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:51.903565 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:51.903572 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:51.903657 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:51.903780 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 18:38:51.903846 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:38:51.907799 4902 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:51.911353 4902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:51.928664 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.923633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46158 (local address 127.18.125.130:37509)
0806 18:38:51.924110 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:51.924122 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:51.924163 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:38:51.924239 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:51.924243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:51.924425 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:51.924608 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:51.924852 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.925944 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.927056 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.927847 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.927983 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.928207 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:51.928363 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 18:38:51.928482 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":124,"threads_started":1}
I20260806 18:38:51.928650 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.923197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:37509 (local address 127.0.0.1:46158)
0806 18:38:51.923718 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:51.923734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:51.923827 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:51.924611 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:51.924614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:51.924634 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:51.924711 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.924717 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.926041 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.926047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.926910 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.926920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.927996 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.928000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.928153 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.928200 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 18:38:51.928316 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 18:38:51.928471 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":28,"thread_start_us":90,"threads_started":1}
I20260806 18:38:51.928694 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.922075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45414 (local address 127.18.125.131:33675)
0806 18:38:51.922668 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:51.922681 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:51.922711 (+ 30us) server_negotiation.cc:407] Connection header received
0806 18:38:51.922998 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:51.923004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:51.923257 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:51.923447 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:51.924036 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.925341 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.927141 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.927762 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.927845 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.928063 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:51.928435 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 18:38:51.928538 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":225,"threads_started":1}
I20260806 18:38:51.928659 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.922787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:42531 (local address 127.0.0.1:37872)
0806 18:38:51.923295 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:51.923310 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:51.923419 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:51.924170 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:51.924173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:51.924192 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:51.924309 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.924320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.925888 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.925891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.927162 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.927170 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.928230 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.928236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.928354 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.928380 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:38:51.928434 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 18:38:51.928522 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":58,"thread_start_us":141,"threads_started":1}
I20260806 18:38:51.929450 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.921851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:33675 (local address 127.0.0.1:45414)
0806 18:38:51.922512 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:51.922585 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:51.922750 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:51.923615 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:51.923621 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:51.923645 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:51.923846 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.923854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.925485 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.925489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.926989 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:51.927001 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.928182 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.928185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:51.928268 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.928302 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 18:38:51.929228 (+ 926us) client_negotiation.cc:240] Negotiation successful
0806 18:38:51.929304 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":36,"thread_start_us":96,"threads_started":1}
I20260806 18:38:51.929798 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:51.923287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37872 (local address 127.18.125.129:42531)
0806 18:38:51.923662 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:51.923669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:51.923691 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:38:51.923783 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:51.923787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:51.923988 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:51.924196 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:51.924446 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.925756 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.927289 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:51.928045 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:51.929282 (+ 1237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:51.929466 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:51.929520 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 18:38:51.929621 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":85,"threads_started":1}
I20260806 18:38:51.932183 5158 tablet_service.cc:1467] Processing CreateTablet for tablet eb4399fe270945b6a628731764a4794a (DEFAULT_TABLE table=test-table [id=35647de4d7f74befb400bc12ee449293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:51.932308 5025 tablet_service.cc:1467] Processing CreateTablet for tablet eb4399fe270945b6a628731764a4794a (DEFAULT_TABLE table=test-table [id=35647de4d7f74befb400bc12ee449293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:51.932308 5290 tablet_service.cc:1467] Processing CreateTablet for tablet eb4399fe270945b6a628731764a4794a (DEFAULT_TABLE table=test-table [id=35647de4d7f74befb400bc12ee449293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:51.933398 5158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb4399fe270945b6a628731764a4794a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:51.933399 5025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb4399fe270945b6a628731764a4794a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:51.933399 5290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb4399fe270945b6a628731764a4794a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:51.940335 5376 tablet_bootstrap.cc:492] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456: Bootstrap starting.
I20260806 18:38:51.941716 5377 tablet_bootstrap.cc:492] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb: Bootstrap starting.
I20260806 18:38:51.942775 5376 tablet_bootstrap.cc:654] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:51.943552 5376 log.cc:824] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:51.943609 5377 tablet_bootstrap.cc:654] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:51.944197 5378 tablet_bootstrap.cc:492] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: Bootstrap starting.
I20260806 18:38:51.944482 5377 log.cc:824] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:51.946215 5376 tablet_bootstrap.cc:492] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456: No bootstrap required, opened a new log
I20260806 18:38:51.946455 5376 ts_tablet_manager.cc:1397] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:38:51.946485 5377 tablet_bootstrap.cc:492] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb: No bootstrap required, opened a new log
I20260806 18:38:51.946676 5377 ts_tablet_manager.cc:1397] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:51.946985 5378 tablet_bootstrap.cc:654] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:51.947891 5378 log.cc:824] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:51.950317 5378 tablet_bootstrap.cc:492] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: No bootstrap required, opened a new log
I20260806 18:38:51.950536 5378 ts_tablet_manager.cc:1397] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:38:51.950493 5376 raft_consensus.cc:348] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33675 } } peers { permanent_uuid: "01b13497a9984bb5b7aad5a52d5981fb" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 42531 } } peers { permanent_uuid: "b7d1d94f486e4f57ac5af5b438d84456" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37509 } }
I20260806 18:38:51.950812 5376 raft_consensus.cc:374] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:51.950877 5376 raft_consensus.cc:729] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7d1d94f486e4f57ac5af5b438d84456, State: Initialized, Role: FOLLOWER
I20260806 18:38:51.951046 5377 raft_consensus.cc:348] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33675 } } peers { permanent_uuid: "01b13497a9984bb5b7aad5a52d5981fb" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 42531 } } peers { permanent_uuid: "b7d1d94f486e4f57ac5af5b438d84456" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37509 } }
I20260806 18:38:51.951324 5377 raft_consensus.cc:374] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:51.951385 5377 raft_consensus.cc:729] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01b13497a9984bb5b7aad5a52d5981fb, State: Initialized, Role: FOLLOWER
I20260806 18:38:51.951467 5376 consensus_queue.cc:260] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33675 } } peers { permanent_uuid: "01b13497a9984bb5b7aad5a52d5981fb" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 42531 } } peers { permanent_uuid: "b7d1d94f486e4f57ac5af5b438d84456" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37509 } }
I20260806 18:38:51.951853 5377 consensus_queue.cc:260] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33675 } } peers { permanent_uuid: "01b13497a9984bb5b7aad5a52d5981fb" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 42531 } } peers { permanent_uuid: "b7d1d94f486e4f57ac5af5b438d84456" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37509 } }
I20260806 18:38:51.952678 5376 ts_tablet_manager.cc:1428] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260806 18:38:51.952833 5377 ts_tablet_manager.cc:1428] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260806 18:38:51.953069 5225 heartbeater.cc:503] Master 127.18.125.190:35653 was elected leader, sending a full tablet report...
I20260806 18:38:51.953174 5093 heartbeater.cc:503] Master 127.18.125.190:35653 was elected leader, sending a full tablet report...
I20260806 18:38:51.955154 5378 raft_consensus.cc:348] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33675 } } peers { permanent_uuid: "01b13497a9984bb5b7aad5a52d5981fb" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 42531 } } peers { permanent_uuid: "b7d1d94f486e4f57ac5af5b438d84456" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37509 } }
I20260806 18:38:51.955461 5378 raft_consensus.cc:374] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:51.955531 5378 raft_consensus.cc:729] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eebca1b5d1d4671a2f26a37c7087cde, State: Initialized, Role: FOLLOWER
I20260806 18:38:51.956022 5378 consensus_queue.cc:260] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33675 } } peers { permanent_uuid: "01b13497a9984bb5b7aad5a52d5981fb" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 42531 } } peers { permanent_uuid: "b7d1d94f486e4f57ac5af5b438d84456" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37509 } }
W20260806 18:38:51.956939 5094 tablet.cc:2369] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:51.957207 5378 ts_tablet_manager.cc:1428] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260806 18:38:51.957582 5358 heartbeater.cc:503] Master 127.18.125.190:35653 was elected leader, sending a full tablet report...
I20260806 18:38:52.067544 5383 raft_consensus.cc:482] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:52.067813 5383 raft_consensus.cc:504] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33675 } } peers { permanent_uuid: "01b13497a9984bb5b7aad5a52d5981fb" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 42531 } } peers { permanent_uuid: "b7d1d94f486e4f57ac5af5b438d84456" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37509 } }
I20260806 18:38:52.069031 5383 leader_election.cc:288] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1eebca1b5d1d4671a2f26a37c7087cde (127.18.125.131:33675), b7d1d94f486e4f57ac5af5b438d84456 (127.18.125.130:37509)
I20260806 18:38:52.074643 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:52.069567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:35213 (local address 127.18.125.131:33675)
0806 18:38:52.069767 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:52.069773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:52.069926 (+ 153us) server_negotiation.cc:407] Connection header received
0806 18:38:52.070123 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:52.070126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:52.070195 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:52.070283 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:52.070685 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:52.071481 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:52.073286 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:52.073930 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:52.074066 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:52.074123 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:52.074452 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 18:38:52.074537 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 18:38:52.074658 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:52.069882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:37509 (local address 127.18.125.129:44703)
0806 18:38:52.070270 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:52.070286 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:52.070393 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:52.070637 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:52.070639 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:52.070662 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:52.070769 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:52.070773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:52.071697 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:52.071700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:52.073145 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:52.073152 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:52.074072 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:52.074075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:52.074269 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:52.074312 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 18:38:52.074393 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:38:52.074480 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":149,"spinlock_wait_cycles":1792,"thread_start_us":116,"threads_started":1}
I20260806 18:38:52.074679 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:52.070029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:44703 (local address 127.18.125.130:37509)
0806 18:38:52.070225 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:52.070231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:52.070384 (+ 153us) server_negotiation.cc:407] Connection header received
0806 18:38:52.070488 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:52.070490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:52.070531 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:52.070592 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:52.070876 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:52.071593 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:52.073286 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:52.073932 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:52.074066 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:52.074138 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:52.074437 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 18:38:52.074523 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 18:38:52.075528 5178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4399fe270945b6a628731764a4794a" candidate_uuid: "01b13497a9984bb5b7aad5a52d5981fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7d1d94f486e4f57ac5af5b438d84456" is_pre_election: true
I20260806 18:38:52.075866 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:52.069408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:33675 (local address 127.18.125.129:35213)
0806 18:38:52.069810 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:52.069833 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:52.069990 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:52.070333 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:52.070347 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:52.070376 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:52.070538 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:52.070545 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:52.071606 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:52.071609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:52.073138 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:52.073152 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:52.074072 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:52.074075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:52.074257 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:52.074308 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 18:38:52.074905 (+ 597us) client_negotiation.cc:240] Negotiation successful
0806 18:38:52.075159 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":114,"thread_start_us":116,"threads_started":1}
I20260806 18:38:52.075858 5178 raft_consensus.cc:2454] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01b13497a9984bb5b7aad5a52d5981fb in term 0.
I20260806 18:38:52.076480 4978 leader_election.cc:302] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01b13497a9984bb5b7aad5a52d5981fb, b7d1d94f486e4f57ac5af5b438d84456; no voters:
I20260806 18:38:52.076515 5311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4399fe270945b6a628731764a4794a" candidate_uuid: "01b13497a9984bb5b7aad5a52d5981fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" is_pre_election: true
I20260806 18:38:52.076717 5311 raft_consensus.cc:2454] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01b13497a9984bb5b7aad5a52d5981fb in term 0.
I20260806 18:38:52.076835 5383 raft_consensus.cc:2781] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:52.076934 5383 raft_consensus.cc:482] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:52.076983 5383 raft_consensus.cc:3037] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:52.078161 5383 raft_consensus.cc:504] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33675 } } peers { permanent_uuid: "01b13497a9984bb5b7aad5a52d5981fb" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 42531 } } peers { permanent_uuid: "b7d1d94f486e4f57ac5af5b438d84456" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37509 } }
I20260806 18:38:52.078518 5383 leader_election.cc:288] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [CANDIDATE]: Term 1 election: Requested vote from peers 1eebca1b5d1d4671a2f26a37c7087cde (127.18.125.131:33675), b7d1d94f486e4f57ac5af5b438d84456 (127.18.125.130:37509)
I20260806 18:38:52.078914 5311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4399fe270945b6a628731764a4794a" candidate_uuid: "01b13497a9984bb5b7aad5a52d5981fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1eebca1b5d1d4671a2f26a37c7087cde"
I20260806 18:38:52.078929 5178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb4399fe270945b6a628731764a4794a" candidate_uuid: "01b13497a9984bb5b7aad5a52d5981fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7d1d94f486e4f57ac5af5b438d84456"
I20260806 18:38:52.079026 5311 raft_consensus.cc:3037] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:52.079068 5178 raft_consensus.cc:3037] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:52.080211 5311 raft_consensus.cc:2454] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01b13497a9984bb5b7aad5a52d5981fb in term 1.
I20260806 18:38:52.080222 5178 raft_consensus.cc:2454] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01b13497a9984bb5b7aad5a52d5981fb in term 1.
I20260806 18:38:52.080535 4978 leader_election.cc:302] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01b13497a9984bb5b7aad5a52d5981fb, 1eebca1b5d1d4671a2f26a37c7087cde; no voters:
I20260806 18:38:52.080717 5383 raft_consensus.cc:2781] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:52.080957 5383 raft_consensus.cc:686] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 1 LEADER]: Becoming Leader. State: Replica: 01b13497a9984bb5b7aad5a52d5981fb, State: Running, Role: LEADER
I20260806 18:38:52.081300 5383 consensus_queue.cc:237] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33675 } } peers { permanent_uuid: "01b13497a9984bb5b7aad5a52d5981fb" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 42531 } } peers { permanent_uuid: "b7d1d94f486e4f57ac5af5b438d84456" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37509 } }
I20260806 18:38:52.084580 4901 catalog_manager.cc:5495] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 01b13497a9984bb5b7aad5a52d5981fb (127.18.125.129). New cstate: current_term: 1 leader_uuid: "01b13497a9984bb5b7aad5a52d5981fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01b13497a9984bb5b7aad5a52d5981fb" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7d1d94f486e4f57ac5af5b438d84456" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37509 } health_report { overall_health: UNKNOWN } } }
I20260806 18:38:52.121658 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:52.117330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37880 (local address 127.18.125.129:42531)
0806 18:38:52.117513 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:52.117520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:52.117539 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:38:52.117636 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:52.117638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:52.117698 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:52.117820 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:52.118097 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:52.118622 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:52.120160 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:52.120397 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:52.120498 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:52.120753 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:52.121208 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:52.121307 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:52.121385 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 18:38:52.121468 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260806 18:38:52.126905 5359 tablet.cc:2369] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:52.128480 5311 raft_consensus.cc:1261] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde [term 1 FOLLOWER]: Refusing update from remote peer 01b13497a9984bb5b7aad5a52d5981fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:52.128480 5178 raft_consensus.cc:1261] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456 [term 1 FOLLOWER]: Refusing update from remote peer 01b13497a9984bb5b7aad5a52d5981fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:52.129316 5387 consensus_queue.cc:1035] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7d1d94f486e4f57ac5af5b438d84456" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 37509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:52.129546 5383 consensus_queue.cc:1035] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1eebca1b5d1d4671a2f26a37c7087cde" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:52.138955 5391 mvcc.cc:205] Tried to move back new op lower bound from 7315626115589693440 to 7315626115407589376. Current Snapshot: MvccSnapshot[applied={T|T < 7315626115589693440}]
I20260806 18:38:52.140260 5392 mvcc.cc:205] Tried to move back new op lower bound from 7315626115589693440 to 7315626115407589376. Current Snapshot: MvccSnapshot[applied={T|T < 7315626115589693440}]
I20260806 18:38:52.146265 4901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:52.146426 4901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:52.149068 4901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72facebca074461a93cb780a09141024: Sending DeleteTablet for 3 replicas of tablet eb4399fe270945b6a628731764a4794a
I20260806 18:38:52.149878 5290 tablet_service.cc:1514] Processing DeleteTablet for tablet eb4399fe270945b6a628731764a4794a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:52 UTC) from {username='slave'} at 127.0.0.1:45414
I20260806 18:38:52.150007 5025 tablet_service.cc:1514] Processing DeleteTablet for tablet eb4399fe270945b6a628731764a4794a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:52 UTC) from {username='slave'} at 127.0.0.1:37872
I20260806 18:38:52.150488 5399 tablet_replica.cc:330] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: stopping tablet replica
I20260806 18:38:52.150535 5400 tablet_replica.cc:330] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb: stopping tablet replica
I20260806 18:38:52.150497 5158 tablet_service.cc:1514] Processing DeleteTablet for tablet eb4399fe270945b6a628731764a4794a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:52 UTC) from {username='slave'} at 127.0.0.1:46158
I20260806 18:38:52.150878 5399 raft_consensus.cc:2229] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:52.150877 5400 raft_consensus.cc:2229] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:38:52.151229 5400 raft_consensus.cc:2258] T eb4399fe270945b6a628731764a4794a P 01b13497a9984bb5b7aad5a52d5981fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:52.151271 5399 raft_consensus.cc:2258] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:52.151798 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 4963
I20260806 18:38:52.155478 5399 ts_tablet_manager.cc:1905] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:52.155826 5401 tablet_replica.cc:330] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456: stopping tablet replica
I20260806 18:38:52.156234 5401 raft_consensus.cc:2229] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:52.156594 5401 raft_consensus.cc:2258] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:52.158120 5401 ts_tablet_manager.cc:1905] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:52.159188 5399 ts_tablet_manager.cc:1918] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:52.159300 5399 log.cc:1196] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/eb4399fe270945b6a628731764a4794a
I20260806 18:38:52.159652 5399 ts_tablet_manager.cc:1939] T eb4399fe270945b6a628731764a4794a P 1eebca1b5d1d4671a2f26a37c7087cde: Deleting consensus metadata
W20260806 18:38:52.160363 4885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:42531 (error 108)
I20260806 18:38:52.161202 5401 ts_tablet_manager.cc:1918] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:52.161237 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 5096
I20260806 18:38:52.161307 5401 log.cc:1196] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/eb4399fe270945b6a628731764a4794a
I20260806 18:38:52.161630 5401 ts_tablet_manager.cc:1939] T eb4399fe270945b6a628731764a4794a P b7d1d94f486e4f57ac5af5b438d84456: Deleting consensus metadata
I20260806 18:38:52.162508 4885 catalog_manager.cc:4841] TS 1eebca1b5d1d4671a2f26a37c7087cde (127.18.125.131:33675): tablet eb4399fe270945b6a628731764a4794a (table test-table [id=35647de4d7f74befb400bc12ee449293]) successfully deleted
I20260806 18:38:52.163578 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:52.163153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:42531 (local address 127.0.0.1:37890)
0806 18:38:52.163400 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:52.163478 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:42531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260806 18:38:52.163743 4885 catalog_manager.cc:4584] TS 01b13497a9984bb5b7aad5a52d5981fb (127.18.125.129:42531): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb4399fe270945b6a628731764a4794a: Network error: Client connection negotiation failed: client connection to 127.18.125.129:42531: connect: Connection refused (error 111)
I20260806 18:38:52.170415 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 5228
I20260806 18:38:52.174067 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:52.170687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:37509 (local address 127.0.0.1:46168)
0806 18:38:52.173868 (+ 3181us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:52.173950 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.130:37509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3050}
I20260806 18:38:52.179369 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 4870
2026-08-06T18:38:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1255 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:52 dist-test-slave-8w9v krb5kdc[5405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:52 dist-test-slave-8w9v krb5kdc[5405](info): set up 2 sockets
Aug 06 18:38:52 dist-test-slave-8w9v krb5kdc[5405](info): commencing operation
krb5kdc: starting...
W20260806 18:38:53.298797 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:38:53 dist-test-slave-8w9v krb5kdc[5405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:38:53.448287 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:36169
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:45495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:36169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:53.566459 5418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:53.566761 5418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:53.566805 5418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:53.570895 5418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:53.570967 5418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:53.570986 5418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:53.571003 5418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:53.571019 5418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:53.576540 5418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:45495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:36169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:36169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:53.577627 5418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:53.578641 5418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:53.587466 5424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:53.587638 5418 server_base.cc:1034] running on GCE node
W20260806 18:38:53.587458 5426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:53.587543 5423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:53.588483 5418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:53.589637 5418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:53.590807 5418 hybrid_clock.cc:648] HybridClock initialized: now 1786041533590788 us; error 32 us; skew 500 ppm
Aug 06 18:38:53 dist-test-slave-8w9v krb5kdc[5405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041533, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:53.596809 5418 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:38:53.597846 5418 webserver.cc:460] Webserver started at http://127.18.125.190:41069/ using document root <none> and password file <none>
I20260806 18:38:53.598469 5418 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:53.598568 5418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:53.598835 5418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:53.601176 5418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "ab355fb3e4e948918764e111dfaefc43"
format_stamp: "Formatted at 2026-08-06 18:38:53 on dist-test-slave-8w9v"
server_key: "f32ccea8136cdbcbbb9de8431cc82dc3"
server_key_iv: "72897eb2bc453b8d2716f0a94fd85a4e"
server_key_version: "encryptionkey@0"
I20260806 18:38:53.601811 5418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "ab355fb3e4e948918764e111dfaefc43"
format_stamp: "Formatted at 2026-08-06 18:38:53 on dist-test-slave-8w9v"
server_key: "f32ccea8136cdbcbbb9de8431cc82dc3"
server_key_iv: "72897eb2bc453b8d2716f0a94fd85a4e"
server_key_version: "encryptionkey@0"
I20260806 18:38:53.605798 5418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:38:53.608502 5434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:53.609879 5418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:38:53.610038 5418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "ab355fb3e4e948918764e111dfaefc43"
format_stamp: "Formatted at 2026-08-06 18:38:53 on dist-test-slave-8w9v"
server_key: "f32ccea8136cdbcbbb9de8431cc82dc3"
server_key_iv: "72897eb2bc453b8d2716f0a94fd85a4e"
server_key_version: "encryptionkey@0"
I20260806 18:38:53.610170 5418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:53.624390 5418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:53.627849 5418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:53.628103 5418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:53.636340 5418 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:36169
I20260806 18:38:53.636348 5486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:36169 every 8 connection(s)
I20260806 18:38:53.637530 5418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:53.640524 5487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:53.646212 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 5418
I20260806 18:38:53.646369 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:38:53.646685 18934 external_mini_cluster.cc:1428] Setting key d906e4823946f1e191b7c26936e207e9
I20260806 18:38:53.646683 5487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43: Bootstrap starting.
I20260806 18:38:53.649561 5487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:53.650532 5487 log.cc:824] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:53.653087 5487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43: No bootstrap required, opened a new log
Aug 06 18:38:53 dist-test-slave-8w9v krb5kdc[5405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:53.657799 5487 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab355fb3e4e948918764e111dfaefc43" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36169 } }
I20260806 18:38:53.658059 5487 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:53.658128 5487 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab355fb3e4e948918764e111dfaefc43, State: Initialized, Role: FOLLOWER
I20260806 18:38:53.658591 5487 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab355fb3e4e948918764e111dfaefc43" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36169 } }
I20260806 18:38:53.658720 5487 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:53.658810 5487 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:53.658938 5487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:53.659958 5487 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab355fb3e4e948918764e111dfaefc43" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36169 } }
I20260806 18:38:53.660385 5487 leader_election.cc:302] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab355fb3e4e948918764e111dfaefc43; no voters:
I20260806 18:38:53.660790 5487 leader_election.cc:288] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:53.660878 5492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:53.661098 5492 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [term 1 LEADER]: Becoming Leader. State: Replica: ab355fb3e4e948918764e111dfaefc43, State: Running, Role: LEADER
I20260806 18:38:53.661502 5492 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab355fb3e4e948918764e111dfaefc43" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36169 } }
I20260806 18:38:53.662278 5487 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:53.663560 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:53.648236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53510 (local address 127.18.125.190:36169)
0806 18:38:53.648705 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:53.648722 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:53.648761 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:38:53.649447 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:53.649464 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:53.649853 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:53.650139 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:53.651175 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:53.652414 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:53.653603 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:53.654086 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:53.654211 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:53.657350 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:53.657356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:53.657371 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:53.657402 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:53.659503 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:53.660026 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:53.660033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:53.660041 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:53.660255 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:53.660571 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:53.660574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:53.660576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:53.661196 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:53.662062 (+ 866us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:53.662195 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 18:38:53.662521 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":146,"threads_started":1}
I20260806 18:38:53.663578 5493 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab355fb3e4e948918764e111dfaefc43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab355fb3e4e948918764e111dfaefc43" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36169 } } }
I20260806 18:38:53.663811 5494 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab355fb3e4e948918764e111dfaefc43. Latest consensus state: current_term: 1 leader_uuid: "ab355fb3e4e948918764e111dfaefc43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab355fb3e4e948918764e111dfaefc43" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36169 } } }
I20260806 18:38:53.663944 5494 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:53.663944 5493 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:53.664914 5501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:53.668015 5501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:53.675180 5501 catalog_manager.cc:1347] Generated new cluster ID: 1c66895884b949fe95a3afcba1af869b
I20260806 18:38:53.675276 5501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:53.689154 5501 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:53.690269 5501 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:53.697460 5501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43: Generated new TSK 0
I20260806 18:38:53.698272 5501 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:38:53.736485 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36169
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:45495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:53.877440 5515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:53.877751 5515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:53.877830 5515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:53.882158 5515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:53.882236 5515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:53.882344 5515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:38:53.888183 5515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:45495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:53.889441 5515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:53.890564 5515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:53.900768 5521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:53.900938 5523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:53.901015 5520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:53.901450 5515 server_base.cc:1034] running on GCE node
I20260806 18:38:53.901970 5515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:53.902712 5515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:53.903925 5515 hybrid_clock.cc:648] HybridClock initialized: now 1786041533903905 us; error 33 us; skew 500 ppm
Aug 06 18:38:53 dist-test-slave-8w9v krb5kdc[5405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041533, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:53.907080 5515 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:38:53.908087 5515 webserver.cc:460] Webserver started at http://127.18.125.129:36069/ using document root <none> and password file <none>
I20260806 18:38:53.908756 5515 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:53.908824 5515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:53.909004 5515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:53.911162 5515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "959f0e4e342943d09f39d396903d0a21"
format_stamp: "Formatted at 2026-08-06 18:38:53 on dist-test-slave-8w9v"
server_key: "5200b283edd8661e1b088c12aefcfa47"
server_key_iv: "180150ab1a6e3db0392fa1dcba5a6b45"
server_key_version: "encryptionkey@0"
I20260806 18:38:53.911718 5515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "959f0e4e342943d09f39d396903d0a21"
format_stamp: "Formatted at 2026-08-06 18:38:53 on dist-test-slave-8w9v"
server_key: "5200b283edd8661e1b088c12aefcfa47"
server_key_iv: "180150ab1a6e3db0392fa1dcba5a6b45"
server_key_version: "encryptionkey@0"
I20260806 18:38:53.916404 5515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260806 18:38:53.919238 5530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:53.921531 5515 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:38:53.921696 5515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "959f0e4e342943d09f39d396903d0a21"
format_stamp: "Formatted at 2026-08-06 18:38:53 on dist-test-slave-8w9v"
server_key: "5200b283edd8661e1b088c12aefcfa47"
server_key_iv: "180150ab1a6e3db0392fa1dcba5a6b45"
server_key_version: "encryptionkey@0"
I20260806 18:38:53.921844 5515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:53.937127 5515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:53.941150 5515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:53.941488 5515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:53.942243 5515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:53.943231 5515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:53.943295 5515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:53.943344 5515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:53.943363 5515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:53.954807 5515 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:44791
I20260806 18:38:53.954833 5645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:44791 every 8 connection(s)
I20260806 18:38:53.955899 5515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:53.956629 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 5515
I20260806 18:38:53.956791 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:53.957168 18934 external_mini_cluster.cc:1428] Setting key 782a98a9c7f24c343122a63884d6d06d
Aug 06 18:38:53 dist-test-slave-8w9v krb5kdc[5405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041533, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
I20260806 18:38:53.971606 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:53.958135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:35071 (local address 127.18.125.190:36169)
0806 18:38:53.958307 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:53.958312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:53.959348 (+ 1036us) server_negotiation.cc:407] Connection header received
0806 18:38:53.960881 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:53.960884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:53.960948 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:53.961053 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:53.962282 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:53.962914 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:53.964307 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:53.964617 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:53.964737 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:53.967387 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:53.967391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:53.967394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:53.967430 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:53.968988 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:53.969590 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:53.969597 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:53.969601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:53.969724 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:53.970114 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:53.970118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:53.970120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:53.970425 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:53.970550 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:53.971169 (+ 619us) server_negotiation.cc:299] Negotiation successful
0806 18:38:53.971336 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 18:38:53.972424 5648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:53.958627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:36169 (local address 127.18.125.129:35071)
0806 18:38:53.959162 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:53.959220 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:53.960620 (+ 1400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:53.961238 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:53.961245 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:53.961611 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:53.962119 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:53.962134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:53.963053 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:53.963056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:53.964163 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:53.964180 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:53.964786 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:53.964790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:53.964982 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:53.965645 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:53.965662 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:53.967214 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:53.969127 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:53.969133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:53.969151 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:53.969433 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:53.969833 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:53.969836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:53.969838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:53.970005 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:53.970615 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:53.970621 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:53.970893 (+ 272us) client_negotiation.cc:769] Sending connection context
0806 18:38:53.971071 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 18:38:53.971362 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":156,"threads_started":1}
I20260806 18:38:53.973875 5646 heartbeater.cc:347] Connected to a master server at 127.18.125.190:36169
I20260806 18:38:53.974170 5646 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:53.974751 5646 heartbeater.cc:511] Master 127.18.125.190:36169 requested a full tablet report, sending...
I20260806 18:38:53.976889 5451 ts_manager.cc:194] Registered new tserver with Master: 959f0e4e342943d09f39d396903d0a21 (127.18.125.129:44791)
I20260806 18:38:53.978818 5451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:35071
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:38:53.993392 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36169
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:45495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:54.123093 5652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:54.123427 5652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:54.123477 5652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:54.128211 5652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:54.128304 5652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:54.128372 5652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:38:54.134743 5652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:45495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:54.136008 5652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:54.137207 5652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:54.147797 5657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:54.147898 5660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:54.147797 5658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:54.148099 5652 server_base.cc:1034] running on GCE node
I20260806 18:38:54.148715 5652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:54.149506 5652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:54.150702 5652 hybrid_clock.cc:648] HybridClock initialized: now 1786041534150672 us; error 45 us; skew 500 ppm
Aug 06 18:38:54 dist-test-slave-8w9v krb5kdc[5405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041534, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:54.154522 5652 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:38:54.155647 5652 webserver.cc:460] Webserver started at http://127.18.125.130:46751/ using document root <none> and password file <none>
I20260806 18:38:54.156426 5652 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:54.156533 5652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:54.156817 5652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:54.159135 5652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "a37a1d15910b44c6b02dd852cf2318e7"
format_stamp: "Formatted at 2026-08-06 18:38:54 on dist-test-slave-8w9v"
server_key: "b311b49c1db22ebbdcc1e9cd7606bace"
server_key_iv: "7753a818a56e8e8fb735218482534a3a"
server_key_version: "encryptionkey@0"
I20260806 18:38:54.159773 5652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "a37a1d15910b44c6b02dd852cf2318e7"
format_stamp: "Formatted at 2026-08-06 18:38:54 on dist-test-slave-8w9v"
server_key: "b311b49c1db22ebbdcc1e9cd7606bace"
server_key_iv: "7753a818a56e8e8fb735218482534a3a"
server_key_version: "encryptionkey@0"
I20260806 18:38:54.164728 5652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 18:38:54.167555 5668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:54.169032 5652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:54.169198 5652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "a37a1d15910b44c6b02dd852cf2318e7"
format_stamp: "Formatted at 2026-08-06 18:38:54 on dist-test-slave-8w9v"
server_key: "b311b49c1db22ebbdcc1e9cd7606bace"
server_key_iv: "7753a818a56e8e8fb735218482534a3a"
server_key_version: "encryptionkey@0"
I20260806 18:38:54.169366 5652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:54.183641 5652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:54.187299 5652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:54.187551 5652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:54.188278 5652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:54.189236 5652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:54.189316 5652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:54.189393 5652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:54.189440 5652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:54.200090 5652 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:39461
I20260806 18:38:54.200110 5784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:39461 every 8 connection(s)
I20260806 18:38:54.201186 5652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:54.201769 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 5652
I20260806 18:38:54.201915 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:54.202232 18934 external_mini_cluster.cc:1428] Setting key 993b9eb637980491f6ebc3e75c2c90e4
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
Aug 06 18:38:54 dist-test-slave-8w9v krb5kdc[5405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041534, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:54.222791 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.204568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:58707 (local address 127.18.125.190:36169)
0806 18:38:54.204732 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:54.204737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:54.205807 (+ 1070us) server_negotiation.cc:407] Connection header received
0806 18:38:54.207333 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:54.207336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:54.207395 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:54.207486 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:54.208874 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.209432 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.210703 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.210941 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.211063 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.217820 (+ 6757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:54.217824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:54.217826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:54.217863 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:54.219836 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:54.220730 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:54.220734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:54.220736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:54.220833 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:54.221308 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:54.221311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:54.221312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:54.221524 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:54.221603 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:54.222509 (+ 906us) server_negotiation.cc:299] Negotiation successful
0806 18:38:54.222652 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:38:54.224733 5787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.205061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:36169 (local address 127.18.125.130:58707)
0806 18:38:54.205641 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:54.205698 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:54.207095 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:54.207686 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:54.207718 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:54.208107 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:54.208697 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.208712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.209590 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.209595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.210550 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.210561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.211129 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.211133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.211323 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.212092 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:54.212112 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:54.217491 (+ 5379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:54.220029 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:54.220041 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:54.220067 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:54.220590 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:54.220951 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:54.220957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:54.220962 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:54.221193 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:54.221652 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:54.221663 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:54.222132 (+ 469us) client_negotiation.cc:769] Sending connection context
0806 18:38:54.222394 (+ 262us) client_negotiation.cc:240] Negotiation successful
0806 18:38:54.222993 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":204,"threads_started":1}
I20260806 18:38:54.226523 5785 heartbeater.cc:347] Connected to a master server at 127.18.125.190:36169
I20260806 18:38:54.226909 5785 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:54.227681 5785 heartbeater.cc:511] Master 127.18.125.190:36169 requested a full tablet report, sending...
I20260806 18:38:54.229202 5451 ts_manager.cc:194] Registered new tserver with Master: a37a1d15910b44c6b02dd852cf2318e7 (127.18.125.130:39461)
I20260806 18:38:54.229902 5451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:58707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:38:54.241860 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36169
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:45495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:54.368247 5790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:54.368604 5790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:54.368695 5790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:54.373109 5790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:54.373193 5790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:54.373260 5790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:38:54.379217 5790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:45495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:36169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:54.380622 5790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:54.381819 5790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:54.392503 5799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:54.392522 5797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:54.392524 5796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:54.392752 5790 server_base.cc:1034] running on GCE node
I20260806 18:38:54.393285 5790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:54.394004 5790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:54.395202 5790 hybrid_clock.cc:648] HybridClock initialized: now 1786041534395191 us; error 38 us; skew 500 ppm
Aug 06 18:38:54 dist-test-slave-8w9v krb5kdc[5405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041534, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:54.398152 5790 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:38:54.399181 5790 webserver.cc:460] Webserver started at http://127.18.125.131:43751/ using document root <none> and password file <none>
I20260806 18:38:54.399778 5790 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:54.399842 5790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:54.400084 5790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:54.402262 5790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "8b6a09d6a112468693a12ee322496b76"
format_stamp: "Formatted at 2026-08-06 18:38:54 on dist-test-slave-8w9v"
server_key: "85e55b83d026d6a7b54442459d09b00f"
server_key_iv: "5e4a1d8afa27b8503a67f037e71f2859"
server_key_version: "encryptionkey@0"
I20260806 18:38:54.402869 5790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "8b6a09d6a112468693a12ee322496b76"
format_stamp: "Formatted at 2026-08-06 18:38:54 on dist-test-slave-8w9v"
server_key: "85e55b83d026d6a7b54442459d09b00f"
server_key_iv: "5e4a1d8afa27b8503a67f037e71f2859"
server_key_version: "encryptionkey@0"
I20260806 18:38:54.406956 5790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:38:54.409654 5806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:54.411003 5790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:38:54.411111 5790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "8b6a09d6a112468693a12ee322496b76"
format_stamp: "Formatted at 2026-08-06 18:38:54 on dist-test-slave-8w9v"
server_key: "85e55b83d026d6a7b54442459d09b00f"
server_key_iv: "5e4a1d8afa27b8503a67f037e71f2859"
server_key_version: "encryptionkey@0"
I20260806 18:38:54.411211 5790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:54.422768 5790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:54.426290 5790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:54.426457 5790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:54.427053 5790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:54.427942 5790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:54.427989 5790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:54.428031 5790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:54.428047 5790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:54.437741 5790 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:33371
I20260806 18:38:54.437767 5923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:33371 every 8 connection(s)
I20260806 18:38:54.438761 5790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:54.440200 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 5790
I20260806 18:38:54.440348 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:54.440712 18934 external_mini_cluster.cc:1428] Setting key afcf71a9fa0cfc8d9f6e686fb7239a25
Aug 06 18:38:54 dist-test-slave-8w9v krb5kdc[5405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041534, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:54.453852 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.440919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:59783 (local address 127.18.125.190:36169)
0806 18:38:54.441075 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:54.441080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:54.442740 (+ 1660us) server_negotiation.cc:407] Connection header received
0806 18:38:54.443734 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:54.443737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:54.443789 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:54.443863 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:54.445206 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.445774 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.447127 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.447364 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.447467 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.450309 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:54.450312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:54.450315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:54.450345 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:54.451641 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:54.452209 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:54.452213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:54.452215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:54.452322 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:54.452697 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:54.452700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:54.452701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:54.452908 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:54.453010 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:54.453549 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 18:38:54.453709 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:38:54.454881 5926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.441437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:36169 (local address 127.18.125.131:59783)
0806 18:38:54.442027 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:54.442082 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:54.443526 (+ 1444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:54.444045 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:54.444053 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:54.444502 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:54.445047 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.445063 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.445943 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.445946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.446967 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.446979 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.447537 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.447540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.447713 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.448485 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:54.448503 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:54.450121 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:54.451757 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:54.451762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:54.451775 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:54.452054 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:54.452428 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:54.452430 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:54.452432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:54.452575 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:54.453012 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:54.453017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:54.453270 (+ 253us) client_negotiation.cc:769] Sending connection context
0806 18:38:54.453437 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 18:38:54.453784 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":179,"threads_started":1}
I20260806 18:38:54.456066 5924 heartbeater.cc:347] Connected to a master server at 127.18.125.190:36169
I20260806 18:38:54.456383 5924 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:54.456940 5924 heartbeater.cc:511] Master 127.18.125.190:36169 requested a full tablet report, sending...
I20260806 18:38:54.458050 5451 ts_manager.cc:194] Registered new tserver with Master: 8b6a09d6a112468693a12ee322496b76 (127.18.125.131:33371)
I20260806 18:38:54.458638 5451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:59783
I20260806 18:38:54.467363 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:54.477474 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.469488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.18.125.190:36169)
0806 18:38:54.469626 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:54.469629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:54.469821 (+ 192us) server_negotiation.cc:407] Connection header received
0806 18:38:54.469927 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:54.469930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:54.469972 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:54.470041 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:54.470736 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.471248 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.472075 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.472340 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.472420 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.474036 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:54.474040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:54.474042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:54.474065 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:54.475475 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:54.476057 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:54.476060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:54.476061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:54.476207 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:54.476527 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:54.476530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:54.476531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:54.476734 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:54.476858 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:54.477139 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 18:38:54.477285 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:38:54.480718 5451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:54.483789 5451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:54.499949 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.493480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45072 (local address 127.18.125.131:33371)
0806 18:38:54.494694 (+ 1214us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:54.494709 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:54.494733 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:38:54.494810 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:54.494818 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:54.495039 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:54.495558 (+ 519us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:54.495731 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.496950 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.498583 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.499193 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.499341 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.499491 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:54.499676 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 18:38:54.499789 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1094,"thread_start_us":145,"threads_started":1}
I20260806 18:38:54.499944 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.493257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:33371 (local address 127.0.0.1:45072)
0806 18:38:54.493655 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:54.493719 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:54.493865 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:54.495219 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:54.495228 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:54.495255 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:54.495427 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.495435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.497095 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.497098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.498462 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.498469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.499334 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.499338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.499449 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.499487 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:38:54.499630 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 18:38:54.499771 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":1,"spinlock_wait_cycles":6016,"thread_start_us":93,"threads_started":1}
I20260806 18:38:54.500497 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.494124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54312 (local address 127.18.125.130:39461)
0806 18:38:54.494777 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:54.494788 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:54.494808 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:38:54.494878 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:54.494882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:54.495059 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:54.495186 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:54.495693 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.496591 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.498377 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.499269 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.499398 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.499532 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:54.500217 (+ 685us) server_negotiation.cc:299] Negotiation successful
0806 18:38:54.500327 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":116,"threads_started":1}
I20260806 18:38:54.500653 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.494014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44791 (local address 127.0.0.1:34924)
0806 18:38:54.494420 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:54.494436 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:54.494527 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:54.495260 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:54.495263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:54.495281 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:54.495525 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.495535 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.497141 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.497143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.498247 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.498257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.499604 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.499608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.499892 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.499956 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 18:38:54.500436 (+ 480us) client_negotiation.cc:240] Negotiation successful
0806 18:38:54.500515 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":193,"spinlock_wait_cycles":2432,"thread_start_us":95,"threads_started":1}
I20260806 18:38:54.501010 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.493653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:39461 (local address 127.0.0.1:54312)
0806 18:38:54.494154 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:54.494168 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:54.494253 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:54.495356 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:54.495358 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:54.495371 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:54.495555 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.495563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.496732 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.496737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.498236 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.498247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.499604 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.499608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.499896 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.499957 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 18:38:54.500032 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:38:54.500886 (+ 854us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":259,"thread_start_us":122,"threads_started":1}
I20260806 18:38:54.501103 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.494241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34924 (local address 127.18.125.129:44791)
0806 18:38:54.494574 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:54.494582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:54.494607 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:38:54.494692 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:54.494698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:54.494934 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:54.495088 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:54.495681 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.496963 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.498429 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.499269 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.499398 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.499532 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:54.500825 (+ 1293us) server_negotiation.cc:299] Negotiation successful
0806 18:38:54.500954 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":113,"threads_started":1}
I20260806 18:38:54.503600 5578 tablet_service.cc:1467] Processing CreateTablet for tablet e5201a1b94804d3e846c62d145781948 (DEFAULT_TABLE table=test-table [id=1cf00098444c4917a4e3f4abd814d809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:54.503589 5857 tablet_service.cc:1467] Processing CreateTablet for tablet e5201a1b94804d3e846c62d145781948 (DEFAULT_TABLE table=test-table [id=1cf00098444c4917a4e3f4abd814d809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:54.504845 5578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5201a1b94804d3e846c62d145781948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:54.504845 5857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5201a1b94804d3e846c62d145781948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:54.506578 5717 tablet_service.cc:1467] Processing CreateTablet for tablet e5201a1b94804d3e846c62d145781948 (DEFAULT_TABLE table=test-table [id=1cf00098444c4917a4e3f4abd814d809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:54.507634 5717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5201a1b94804d3e846c62d145781948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:54.513809 5943 tablet_bootstrap.cc:492] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76: Bootstrap starting.
I20260806 18:38:54.514071 5942 tablet_bootstrap.cc:492] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21: Bootstrap starting.
I20260806 18:38:54.514683 5944 tablet_bootstrap.cc:492] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7: Bootstrap starting.
I20260806 18:38:54.517150 5944 tablet_bootstrap.cc:654] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:54.517179 5943 tablet_bootstrap.cc:654] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:54.518425 5942 tablet_bootstrap.cc:654] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:54.518698 5944 log.cc:824] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:54.519040 5943 log.cc:824] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:54.519289 5942 log.cc:824] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:54.521155 5944 tablet_bootstrap.cc:492] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7: No bootstrap required, opened a new log
I20260806 18:38:54.521409 5944 ts_tablet_manager.cc:1397] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 18:38:54.521435 5943 tablet_bootstrap.cc:492] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76: No bootstrap required, opened a new log
I20260806 18:38:54.521430 5942 tablet_bootstrap.cc:492] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21: No bootstrap required, opened a new log
I20260806 18:38:54.521633 5943 ts_tablet_manager.cc:1397] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 18:38:54.521633 5942 ts_tablet_manager.cc:1397] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260806 18:38:54.525978 5944 raft_consensus.cc:348] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6a09d6a112468693a12ee322496b76" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33371 } } peers { permanent_uuid: "959f0e4e342943d09f39d396903d0a21" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44791 } } peers { permanent_uuid: "a37a1d15910b44c6b02dd852cf2318e7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39461 } }
I20260806 18:38:54.526458 5944 raft_consensus.cc:374] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:54.526523 5944 raft_consensus.cc:729] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37a1d15910b44c6b02dd852cf2318e7, State: Initialized, Role: FOLLOWER
I20260806 18:38:54.526304 5943 raft_consensus.cc:348] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6a09d6a112468693a12ee322496b76" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33371 } } peers { permanent_uuid: "959f0e4e342943d09f39d396903d0a21" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44791 } } peers { permanent_uuid: "a37a1d15910b44c6b02dd852cf2318e7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39461 } }
I20260806 18:38:54.526594 5943 raft_consensus.cc:374] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:54.526667 5943 raft_consensus.cc:729] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b6a09d6a112468693a12ee322496b76, State: Initialized, Role: FOLLOWER
I20260806 18:38:54.526883 5942 raft_consensus.cc:348] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6a09d6a112468693a12ee322496b76" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33371 } } peers { permanent_uuid: "959f0e4e342943d09f39d396903d0a21" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44791 } } peers { permanent_uuid: "a37a1d15910b44c6b02dd852cf2318e7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39461 } }
I20260806 18:38:54.527076 5944 consensus_queue.cc:260] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6a09d6a112468693a12ee322496b76" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33371 } } peers { permanent_uuid: "959f0e4e342943d09f39d396903d0a21" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44791 } } peers { permanent_uuid: "a37a1d15910b44c6b02dd852cf2318e7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39461 } }
I20260806 18:38:54.527182 5943 consensus_queue.cc:260] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6a09d6a112468693a12ee322496b76" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33371 } } peers { permanent_uuid: "959f0e4e342943d09f39d396903d0a21" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44791 } } peers { permanent_uuid: "a37a1d15910b44c6b02dd852cf2318e7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39461 } }
I20260806 18:38:54.527346 5942 raft_consensus.cc:374] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:54.527437 5942 raft_consensus.cc:729] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959f0e4e342943d09f39d396903d0a21, State: Initialized, Role: FOLLOWER
I20260806 18:38:54.527878 5942 consensus_queue.cc:260] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6a09d6a112468693a12ee322496b76" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33371 } } peers { permanent_uuid: "959f0e4e342943d09f39d396903d0a21" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44791 } } peers { permanent_uuid: "a37a1d15910b44c6b02dd852cf2318e7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39461 } }
I20260806 18:38:54.528298 5944 ts_tablet_manager.cc:1428] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 18:38:54.528329 5943 ts_tablet_manager.cc:1428] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 18:38:54.528664 5785 heartbeater.cc:503] Master 127.18.125.190:36169 was elected leader, sending a full tablet report...
I20260806 18:38:54.528702 5924 heartbeater.cc:503] Master 127.18.125.190:36169 was elected leader, sending a full tablet report...
I20260806 18:38:54.530005 5942 ts_tablet_manager.cc:1428] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21: Time spent starting tablet: real 0.008s user 0.002s sys 0.006s
I20260806 18:38:54.530421 5646 heartbeater.cc:503] Master 127.18.125.190:36169 was elected leader, sending a full tablet report...
I20260806 18:38:54.532714 5948 raft_consensus.cc:482] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:54.532884 5948 raft_consensus.cc:504] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6a09d6a112468693a12ee322496b76" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33371 } } peers { permanent_uuid: "959f0e4e342943d09f39d396903d0a21" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44791 } } peers { permanent_uuid: "a37a1d15910b44c6b02dd852cf2318e7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39461 } }
I20260806 18:38:54.534277 5948 leader_election.cc:288] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b6a09d6a112468693a12ee322496b76 (127.18.125.131:33371), 959f0e4e342943d09f39d396903d0a21 (127.18.125.129:44791)
I20260806 18:38:54.538563 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.535196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:47889 (local address 127.18.125.131:33371)
0806 18:38:54.535321 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:54.535326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:54.535347 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:38:54.535387 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:54.535389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:54.535427 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:54.535491 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:54.535849 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.536473 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.537560 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.537973 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.538105 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.538158 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:54.538374 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 18:38:54.538442 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:38:54.538563 5787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.534469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:33371 (local address 127.18.125.130:47889)
0806 18:38:54.534618 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:54.534637 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:54.534747 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:54.535530 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:54.535533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:54.535551 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:54.535723 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.535732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.536589 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.536594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.537409 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.537436 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.538100 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.538103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.538209 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.538265 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 18:38:54.538355 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 18:38:54.538434 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":51}
I20260806 18:38:54.538726 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.534862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:35207 (local address 127.18.125.129:44791)
0806 18:38:54.535003 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:54.535008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:54.535221 (+ 213us) server_negotiation.cc:407] Connection header received
0806 18:38:54.535376 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:54.535379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:54.535433 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:54.535525 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:54.535925 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.536528 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.537627 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.538068 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.538215 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.538281 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:54.538504 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 18:38:54.538584 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:38:54.538743 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.534771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44791 (local address 127.18.125.130:35207)
0806 18:38:54.535112 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:54.535124 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:54.535266 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:54.535639 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:54.535645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:54.535662 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:54.535824 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.535831 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.536656 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.536660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.537486 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:54.537495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.538215 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.538220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:54.538358 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.538390 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 18:38:54.538489 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 18:38:54.538567 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":5,"thread_start_us":155,"threads_started":1}
I20260806 18:38:54.539669 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5201a1b94804d3e846c62d145781948" candidate_uuid: "a37a1d15910b44c6b02dd852cf2318e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b6a09d6a112468693a12ee322496b76" is_pre_election: true
I20260806 18:38:54.539665 5598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5201a1b94804d3e846c62d145781948" candidate_uuid: "a37a1d15910b44c6b02dd852cf2318e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959f0e4e342943d09f39d396903d0a21" is_pre_election: true
I20260806 18:38:54.539988 5877 raft_consensus.cc:2454] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a37a1d15910b44c6b02dd852cf2318e7 in term 0.
I20260806 18:38:54.539988 5598 raft_consensus.cc:2454] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a37a1d15910b44c6b02dd852cf2318e7 in term 0.
I20260806 18:38:54.540522 5672 leader_election.cc:302] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b6a09d6a112468693a12ee322496b76, a37a1d15910b44c6b02dd852cf2318e7; no voters:
I20260806 18:38:54.540771 5948 raft_consensus.cc:2781] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:54.540860 5948 raft_consensus.cc:482] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:54.540920 5948 raft_consensus.cc:3037] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:54.542114 5948 raft_consensus.cc:504] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6a09d6a112468693a12ee322496b76" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33371 } } peers { permanent_uuid: "959f0e4e342943d09f39d396903d0a21" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44791 } } peers { permanent_uuid: "a37a1d15910b44c6b02dd852cf2318e7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39461 } }
I20260806 18:38:54.542505 5948 leader_election.cc:288] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [CANDIDATE]: Term 1 election: Requested vote from peers 8b6a09d6a112468693a12ee322496b76 (127.18.125.131:33371), 959f0e4e342943d09f39d396903d0a21 (127.18.125.129:44791)
I20260806 18:38:54.542992 5598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5201a1b94804d3e846c62d145781948" candidate_uuid: "a37a1d15910b44c6b02dd852cf2318e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959f0e4e342943d09f39d396903d0a21"
I20260806 18:38:54.542989 5877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5201a1b94804d3e846c62d145781948" candidate_uuid: "a37a1d15910b44c6b02dd852cf2318e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b6a09d6a112468693a12ee322496b76"
I20260806 18:38:54.543125 5877 raft_consensus.cc:3037] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:54.543125 5598 raft_consensus.cc:3037] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:54.544502 5877 raft_consensus.cc:2454] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37a1d15910b44c6b02dd852cf2318e7 in term 1.
I20260806 18:38:54.544502 5598 raft_consensus.cc:2454] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37a1d15910b44c6b02dd852cf2318e7 in term 1.
I20260806 18:38:54.544888 5672 leader_election.cc:302] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 959f0e4e342943d09f39d396903d0a21, a37a1d15910b44c6b02dd852cf2318e7; no voters:
I20260806 18:38:54.545183 5948 raft_consensus.cc:2781] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:54.545439 5948 raft_consensus.cc:686] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 1 LEADER]: Becoming Leader. State: Replica: a37a1d15910b44c6b02dd852cf2318e7, State: Running, Role: LEADER
I20260806 18:38:54.545778 5948 consensus_queue.cc:237] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6a09d6a112468693a12ee322496b76" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33371 } } peers { permanent_uuid: "959f0e4e342943d09f39d396903d0a21" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44791 } } peers { permanent_uuid: "a37a1d15910b44c6b02dd852cf2318e7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39461 } }
I20260806 18:38:54.549082 5451 catalog_manager.cc:5495] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a37a1d15910b44c6b02dd852cf2318e7 (127.18.125.130). New cstate: current_term: 1 leader_uuid: "a37a1d15910b44c6b02dd852cf2318e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6a09d6a112468693a12ee322496b76" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "959f0e4e342943d09f39d396903d0a21" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a37a1d15910b44c6b02dd852cf2318e7" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 39461 } health_report { overall_health: HEALTHY } } }
I20260806 18:38:54.567063 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.563080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54328 (local address 127.18.125.130:39461)
0806 18:38:54.563209 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:54.563214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:54.563228 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:38:54.563427 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:54.563432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:54.563566 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:54.563675 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:54.563912 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.564530 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.565526 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.565746 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.565828 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.566099 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:54.566527 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:54.566641 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:54.566802 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 18:38:54.566841 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:38:54.573738 5877 raft_consensus.cc:1261] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76 [term 1 FOLLOWER]: Refusing update from remote peer a37a1d15910b44c6b02dd852cf2318e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:54.573774 5598 raft_consensus.cc:1261] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21 [term 1 FOLLOWER]: Refusing update from remote peer a37a1d15910b44c6b02dd852cf2318e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:54.574653 5948 consensus_queue.cc:1035] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b6a09d6a112468693a12ee322496b76" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:54.574913 5952 consensus_queue.cc:1035] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959f0e4e342943d09f39d396903d0a21" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:54.584746 5957 mvcc.cc:205] Tried to move back new op lower bound from 7315626125605494784 to 7315626125502492672. Current Snapshot: MvccSnapshot[applied={T|T < 7315626125605494784}]
I20260806 18:38:54.584977 5956 mvcc.cc:205] Tried to move back new op lower bound from 7315626125605494784 to 7315626125502492672. Current Snapshot: MvccSnapshot[applied={T|T < 7315626125605494784}]
I20260806 18:38:54.592943 5451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:54.593164 5451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:54.596001 5451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab355fb3e4e948918764e111dfaefc43: Sending DeleteTablet for 3 replicas of tablet e5201a1b94804d3e846c62d145781948
I20260806 18:38:54.597041 5717 tablet_service.cc:1514] Processing DeleteTablet for tablet e5201a1b94804d3e846c62d145781948 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:54 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:54312
I20260806 18:38:54.597080 5857 tablet_service.cc:1514] Processing DeleteTablet for tablet e5201a1b94804d3e846c62d145781948 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:54 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:45072
I20260806 18:38:54.597108 5578 tablet_service.cc:1514] Processing DeleteTablet for tablet e5201a1b94804d3e846c62d145781948 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:54 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:34924
I20260806 18:38:54.597692 5965 tablet_replica.cc:330] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21: stopping tablet replica
I20260806 18:38:54.598078 5965 raft_consensus.cc:2229] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:54.598129 5966 tablet_replica.cc:330] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7: stopping tablet replica
I20260806 18:38:54.598333 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 5515
I20260806 18:38:54.598436 5966 raft_consensus.cc:2229] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:38:54.598453 5965 raft_consensus.cc:2258] T e5201a1b94804d3e846c62d145781948 P 959f0e4e342943d09f39d396903d0a21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:54.598887 5967 tablet_replica.cc:330] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76: stopping tablet replica
I20260806 18:38:54.599155 5966 raft_consensus.cc:2258] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:54.599186 5967 raft_consensus.cc:2229] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:54.599493 5967 raft_consensus.cc:2258] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:54.601200 5966 ts_tablet_manager.cc:1905] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:54.601212 5967 ts_tablet_manager.cc:1905] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:54.604295 5967 ts_tablet_manager.cc:1918] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:54.604410 5967 log.cc:1196] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/e5201a1b94804d3e846c62d145781948
I20260806 18:38:54.604790 5967 ts_tablet_manager.cc:1939] T e5201a1b94804d3e846c62d145781948 P 8b6a09d6a112468693a12ee322496b76: Deleting consensus metadata
I20260806 18:38:54.604849 5966 ts_tablet_manager.cc:1918] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:54.604955 5966 log.cc:1196] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/e5201a1b94804d3e846c62d145781948
I20260806 18:38:54.605273 5966 ts_tablet_manager.cc:1939] T e5201a1b94804d3e846c62d145781948 P a37a1d15910b44c6b02dd852cf2318e7: Deleting consensus metadata
I20260806 18:38:54.606372 5438 catalog_manager.cc:4841] TS 8b6a09d6a112468693a12ee322496b76 (127.18.125.131:33371): tablet e5201a1b94804d3e846c62d145781948 (table test-table [id=1cf00098444c4917a4e3f4abd814d809]) successfully deleted
I20260806 18:38:54.606448 5437 catalog_manager.cc:4841] TS a37a1d15910b44c6b02dd852cf2318e7 (127.18.125.130:39461): tablet e5201a1b94804d3e846c62d145781948 (table test-table [id=1cf00098444c4917a4e3f4abd814d809]) successfully deleted
W20260806 18:38:54.609993 5438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:44791 (error 108)
I20260806 18:38:54.610980 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 5652
I20260806 18:38:54.611241 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.610864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44791 (local address 127.0.0.1:34930)
0806 18:38:54.611042 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:54.611130 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:44791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 18:38:54.613483 5438 catalog_manager.cc:4584] TS 959f0e4e342943d09f39d396903d0a21 (127.18.125.129:44791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5201a1b94804d3e846c62d145781948: Network error: Client connection negotiation failed: client connection to 127.18.125.129:44791: connect: Connection refused (error 111)
I20260806 18:38:54.619546 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 5790
I20260806 18:38:54.627230 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 5418
2026-08-06T18:38:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2448 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T18:38:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:54Z Disabled control of system clock
I20260806 18:38:54.659744 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:40923
--webserver_interface=127.18.125.190
--webserver_port=0
--builtin_ntp_servers=127.18.125.148:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:40923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:54.780928 5972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:54.781232 5972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:54.781271 5972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:54.785668 5972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:54.785741 5972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:54.785758 5972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:54.785776 5972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:54.785799 5972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:54.791152 5972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:40923
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:40923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:54.792407 5972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:54.793839 5972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:54.803591 5977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:54.803596 5978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:54.803594 5980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:54.804085 5972 server_base.cc:1034] running on GCE node
I20260806 18:38:54.804761 5972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:54.806016 5972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:54.807237 5972 hybrid_clock.cc:648] HybridClock initialized: now 1786041534807207 us; error 50 us; skew 500 ppm
I20260806 18:38:54.809623 5972 webserver.cc:460] Webserver started at http://127.18.125.190:44773/ using document root <none> and password file <none>
I20260806 18:38:54.810304 5972 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:54.810444 5972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:54.810711 5972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:54.813077 5972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "71a26f9f21e142b19c305c019ccdb7d8"
format_stamp: "Formatted at 2026-08-06 18:38:54 on dist-test-slave-8w9v"
server_key: "4183b8ad0bfb35a255ae5609835825c6"
server_key_iv: "4dbbb50a307b6c008d936cfa52eb8f34"
server_key_version: "encryptionkey@0"
I20260806 18:38:54.813741 5972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "71a26f9f21e142b19c305c019ccdb7d8"
format_stamp: "Formatted at 2026-08-06 18:38:54 on dist-test-slave-8w9v"
server_key: "4183b8ad0bfb35a255ae5609835825c6"
server_key_iv: "4dbbb50a307b6c008d936cfa52eb8f34"
server_key_version: "encryptionkey@0"
I20260806 18:38:54.818933 5972 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:54.822603 5986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:54.825439 5972 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 18:38:54.825677 5972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "71a26f9f21e142b19c305c019ccdb7d8"
format_stamp: "Formatted at 2026-08-06 18:38:54 on dist-test-slave-8w9v"
server_key: "4183b8ad0bfb35a255ae5609835825c6"
server_key_iv: "4dbbb50a307b6c008d936cfa52eb8f34"
server_key_version: "encryptionkey@0"
I20260806 18:38:54.825874 5972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:54.843361 5972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:54.844358 5972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:54.844590 5972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:54.854094 5972 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:40923
I20260806 18:38:54.854249 6039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:40923 every 8 connection(s)
I20260806 18:38:54.855227 5972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:54.857522 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 5972
I20260806 18:38:54.857630 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:38:54.857973 18934 external_mini_cluster.cc:1428] Setting key 6ba9928721d11f887f847c23a9720fec
I20260806 18:38:54.858963 6040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:54.866812 6040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8: Bootstrap starting.
I20260806 18:38:54.869405 6040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:54.869486 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:54.859515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56204 (local address 127.18.125.190:40923)
0806 18:38:54.860207 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:54.860224 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:54.860262 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:38:54.860968 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:54.860985 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:54.861579 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:54.862095 (+ 516us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:54.862580 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.864348 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.865628 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:54.866360 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:54.866497 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:54.867605 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:54.867610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:54.867624 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:54.868038 (+ 414us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:54.868098 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:54.868118 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:54.868256 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:54.868423 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 18:38:54.868749 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":150,"threads_started":1}
I20260806 18:38:54.870388 6040 log.cc:824] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:54.872931 6040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8: No bootstrap required, opened a new log
I20260806 18:38:54.877055 6040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a26f9f21e142b19c305c019ccdb7d8" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40923 } }
I20260806 18:38:54.877337 6040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:54.877388 6040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71a26f9f21e142b19c305c019ccdb7d8, State: Initialized, Role: FOLLOWER
I20260806 18:38:54.877797 6040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a26f9f21e142b19c305c019ccdb7d8" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40923 } }
I20260806 18:38:54.877920 6040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:54.878018 6040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:54.878121 6040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:54.879364 6040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a26f9f21e142b19c305c019ccdb7d8" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40923 } }
I20260806 18:38:54.879781 6040 leader_election.cc:302] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71a26f9f21e142b19c305c019ccdb7d8; no voters:
I20260806 18:38:54.880209 6040 leader_election.cc:288] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:54.880424 6046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:54.880698 6046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [term 1 LEADER]: Becoming Leader. State: Replica: 71a26f9f21e142b19c305c019ccdb7d8, State: Running, Role: LEADER
I20260806 18:38:54.881228 6046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a26f9f21e142b19c305c019ccdb7d8" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40923 } }
I20260806 18:38:54.881697 6040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:54.883237 6047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71a26f9f21e142b19c305c019ccdb7d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a26f9f21e142b19c305c019ccdb7d8" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40923 } } }
I20260806 18:38:54.883396 6047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:54.883842 6056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:54.883792 6048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71a26f9f21e142b19c305c019ccdb7d8. Latest consensus state: current_term: 1 leader_uuid: "71a26f9f21e142b19c305c019ccdb7d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71a26f9f21e142b19c305c019ccdb7d8" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 40923 } } }
I20260806 18:38:54.883975 6048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:54.887056 6056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:54.893273 6056 catalog_manager.cc:1347] Generated new cluster ID: ae931b80fdf547209f8b85801c886738
I20260806 18:38:54.893362 6056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:54.901216 6056 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:54.902570 6056 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:54.909235 6056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8: Generated new TSK 0
I20260806 18:38:54.910120 6056 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:54.914587 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40923
--builtin_ntp_servers=127.18.125.148:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:55.040508 6066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:55.040796 6066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:55.040839 6066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:55.045264 6066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:55.045347 6066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:55.045437 6066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:38:55.050696 6066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:55.052042 6066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:55.053645 6066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:55.064332 6071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:55.064277 6072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:55.064277 6075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:55.064316 6066 server_base.cc:1034] running on GCE node
I20260806 18:38:55.065069 6066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:55.065795 6066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:55.066995 6066 hybrid_clock.cc:648] HybridClock initialized: now 1786041535066981 us; error 25 us; skew 500 ppm
I20260806 18:38:55.068912 6066 webserver.cc:460] Webserver started at http://127.18.125.129:39789/ using document root <none> and password file <none>
I20260806 18:38:55.069676 6066 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:55.069782 6066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:55.070070 6066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:55.072422 6066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "30d20ab440a548e4b439aeda253973fe"
format_stamp: "Formatted at 2026-08-06 18:38:55 on dist-test-slave-8w9v"
server_key: "8c70474afed505a0da1d274df5832c70"
server_key_iv: "7f33a0ac3ca037e4f43e8d5f1f2f4772"
server_key_version: "encryptionkey@0"
I20260806 18:38:55.072965 6066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "30d20ab440a548e4b439aeda253973fe"
format_stamp: "Formatted at 2026-08-06 18:38:55 on dist-test-slave-8w9v"
server_key: "8c70474afed505a0da1d274df5832c70"
server_key_iv: "7f33a0ac3ca037e4f43e8d5f1f2f4772"
server_key_version: "encryptionkey@0"
I20260806 18:38:55.076960 6066 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:38:55.079510 6081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:55.080847 6066 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:38:55.080978 6066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "30d20ab440a548e4b439aeda253973fe"
format_stamp: "Formatted at 2026-08-06 18:38:55 on dist-test-slave-8w9v"
server_key: "8c70474afed505a0da1d274df5832c70"
server_key_iv: "7f33a0ac3ca037e4f43e8d5f1f2f4772"
server_key_version: "encryptionkey@0"
I20260806 18:38:55.081099 6066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:55.093595 6066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:55.094403 6066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:55.094635 6066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:55.095322 6066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:55.096522 6066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:55.096596 6066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:55.096678 6066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:55.096721 6066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:55.107190 6066 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:44149
I20260806 18:38:55.107280 6199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:44149 every 8 connection(s)
I20260806 18:38:55.108316 6066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:55.113013 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 6066
I20260806 18:38:55.113125 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:55.113466 18934 external_mini_cluster.cc:1428] Setting key a65a6d60d4ff2f8af0370d67dfa9065a
I20260806 18:38:55.115530 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40923
--builtin_ntp_servers=127.18.125.148:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:55.118837 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.110406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:40597 (local address 127.18.125.190:40923)
0806 18:38:55.110565 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:55.110569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:55.111695 (+ 1126us) server_negotiation.cc:407] Connection header received
0806 18:38:55.113346 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:55.113349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:55.113422 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:55.113534 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:55.114629 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.115308 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.116682 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.116912 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.117040 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.118298 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:55.118302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:55.118304 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:55.118317 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:55.118338 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:55.118345 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:55.118425 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:55.118601 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 18:38:55.118684 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:38:55.119894 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.110822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:40923 (local address 127.18.125.129:40597)
0806 18:38:55.111530 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:55.111576 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:55.113053 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:55.113732 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:55.113739 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:55.113851 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:55.114482 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.114497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.115459 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.115462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.116538 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.116551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.117091 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.117096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.117289 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.117994 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:55.117996 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:55.118041 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:55.118115 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:55.118428 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:55.118441 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:55.118448 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:38:55.118589 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 18:38:55.118919 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":127,"threads_started":1}
I20260806 18:38:55.121407 6200 heartbeater.cc:347] Connected to a master server at 127.18.125.190:40923
I20260806 18:38:55.121680 6200 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:55.122278 6200 heartbeater.cc:511] Master 127.18.125.190:40923 requested a full tablet report, sending...
I20260806 18:38:55.124218 6001 ts_manager.cc:194] Registered new tserver with Master: 30d20ab440a548e4b439aeda253973fe (127.18.125.129:44149)
I20260806 18:38:55.126192 6001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.129:40597
W20260806 18:38:55.235644 6203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:55.235980 6203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:55.236025 6203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:55.240243 6203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:55.240310 6203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:55.240375 6203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:38:55.245604 6203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:55.246874 6203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:55.248418 6203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:55.259318 6208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:55.259338 6209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:55.259369 6211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:55.259650 6203 server_base.cc:1034] running on GCE node
I20260806 18:38:55.260789 6203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:55.261580 6203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:55.262776 6203 hybrid_clock.cc:648] HybridClock initialized: now 1786041535262731 us; error 62 us; skew 500 ppm
I20260806 18:38:55.265156 6203 webserver.cc:460] Webserver started at http://127.18.125.130:43579/ using document root <none> and password file <none>
I20260806 18:38:55.265852 6203 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:55.265930 6203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:55.266165 6203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:55.268728 6203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "64eb35b851854ef787877d4c85b852a6"
format_stamp: "Formatted at 2026-08-06 18:38:55 on dist-test-slave-8w9v"
server_key: "ee3e2e97693682f03619a09d2ca8b48c"
server_key_iv: "26d1b1f048b5dd311d72d3aaa54f498a"
server_key_version: "encryptionkey@0"
I20260806 18:38:55.269433 6203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "64eb35b851854ef787877d4c85b852a6"
format_stamp: "Formatted at 2026-08-06 18:38:55 on dist-test-slave-8w9v"
server_key: "ee3e2e97693682f03619a09d2ca8b48c"
server_key_iv: "26d1b1f048b5dd311d72d3aaa54f498a"
server_key_version: "encryptionkey@0"
I20260806 18:38:55.274813 6203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:55.278256 6217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:55.279886 6203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:38:55.280068 6203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "64eb35b851854ef787877d4c85b852a6"
format_stamp: "Formatted at 2026-08-06 18:38:55 on dist-test-slave-8w9v"
server_key: "ee3e2e97693682f03619a09d2ca8b48c"
server_key_iv: "26d1b1f048b5dd311d72d3aaa54f498a"
server_key_version: "encryptionkey@0"
I20260806 18:38:55.280249 6203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:55.306442 6203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:55.307348 6203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:55.307592 6203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:55.308324 6203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:55.309409 6203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:55.309477 6203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:55.309551 6203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:55.309589 6203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:55.320612 6203 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:33267
I20260806 18:38:55.320690 6334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:33267 every 8 connection(s)
I20260806 18:38:55.321669 6203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:55.324019 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 6203
I20260806 18:38:55.324200 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:55.324522 18934 external_mini_cluster.cc:1428] Setting key c41404bd431ca8da1c338ab706829ea6
I20260806 18:38:55.326889 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40923
--builtin_ntp_servers=127.18.125.148:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:55.333976 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.323792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:49203 (local address 127.18.125.190:40923)
0806 18:38:55.323981 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:55.323987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:55.325274 (+ 1287us) server_negotiation.cc:407] Connection header received
0806 18:38:55.327082 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:55.327088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:55.327151 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:55.327239 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:55.328708 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.329553 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.331390 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.331713 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.331787 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.333397 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:55.333401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:55.333404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:55.333422 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:55.333457 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:55.333467 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:55.333511 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:55.333736 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 18:38:55.333806 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:38:55.335683 6337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.324338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:40923 (local address 127.18.125.130:49203)
0806 18:38:55.325105 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:55.325163 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:55.326808 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:55.327490 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:55.327503 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:55.327633 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:55.328492 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.328518 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.329739 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.329743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.331245 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.331258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.331997 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.332001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.332232 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.333047 (+ 815us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:55.333049 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:55.333108 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:55.333204 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:55.333578 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:55.333588 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:55.333593 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:38:55.334240 (+ 647us) client_negotiation.cc:240] Negotiation successful
0806 18:38:55.334592 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":145,"threads_started":1}
I20260806 18:38:55.337096 6335 heartbeater.cc:347] Connected to a master server at 127.18.125.190:40923
I20260806 18:38:55.337347 6335 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:55.338007 6335 heartbeater.cc:511] Master 127.18.125.190:40923 requested a full tablet report, sending...
I20260806 18:38:55.339346 6001 ts_manager.cc:194] Registered new tserver with Master: 64eb35b851854ef787877d4c85b852a6 (127.18.125.130:33267)
I20260806 18:38:55.340224 6001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.130:49203
W20260806 18:38:55.453666 6338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:55.453940 6338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:55.453980 6338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:55.457991 6338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:55.458055 6338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:55.458112 6338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:38:55.463340 6338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:40923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:55.464643 6338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:55.466095 6338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:55.476977 6344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:55.477219 6346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:55.477357 6343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:55.477459 6338 server_base.cc:1034] running on GCE node
I20260806 18:38:55.477996 6338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:55.478679 6338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:55.479858 6338 hybrid_clock.cc:648] HybridClock initialized: now 1786041535479843 us; error 29 us; skew 500 ppm
I20260806 18:38:55.482080 6338 webserver.cc:460] Webserver started at http://127.18.125.131:37141/ using document root <none> and password file <none>
I20260806 18:38:55.482744 6338 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:55.482815 6338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:55.482998 6338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:55.485546 6338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "d9d6255684e34a30bb6ad220cb7a2ace"
format_stamp: "Formatted at 2026-08-06 18:38:55 on dist-test-slave-8w9v"
server_key: "1409a851ac751136eb5b372c9f9e3541"
server_key_iv: "c832d09d56200f7d4c553c7bec33863b"
server_key_version: "encryptionkey@0"
I20260806 18:38:55.486163 6338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "d9d6255684e34a30bb6ad220cb7a2ace"
format_stamp: "Formatted at 2026-08-06 18:38:55 on dist-test-slave-8w9v"
server_key: "1409a851ac751136eb5b372c9f9e3541"
server_key_iv: "c832d09d56200f7d4c553c7bec33863b"
server_key_version: "encryptionkey@0"
I20260806 18:38:55.490202 6338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:38:55.492835 6353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:55.494375 6338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:55.494534 6338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "d9d6255684e34a30bb6ad220cb7a2ace"
format_stamp: "Formatted at 2026-08-06 18:38:55 on dist-test-slave-8w9v"
server_key: "1409a851ac751136eb5b372c9f9e3541"
server_key_iv: "c832d09d56200f7d4c553c7bec33863b"
server_key_version: "encryptionkey@0"
I20260806 18:38:55.494663 6338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:55.505106 6338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:55.505867 6338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:55.506088 6338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:55.506728 6338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:55.507737 6338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:55.507787 6338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:55.507831 6338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:55.507887 6338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:55.517669 6338 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:33929
I20260806 18:38:55.517701 6471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:33929 every 8 connection(s)
I20260806 18:38:55.518642 6338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:55.526355 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 6338
I20260806 18:38:55.526460 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:55.526789 18934 external_mini_cluster.cc:1428] Setting key 3e23827b865f3b1cc1711d06b5b41f6b
I20260806 18:38:55.529023 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.520728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:47839 (local address 127.18.125.190:40923)
0806 18:38:55.520883 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:55.520887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:55.522037 (+ 1150us) server_negotiation.cc:407] Connection header received
0806 18:38:55.523534 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:55.523537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:55.523598 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:55.523696 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:55.524849 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.525433 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.526773 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.527090 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.527201 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.528542 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:55.528545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:55.528547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:55.528559 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:55.528580 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:55.528588 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:55.528651 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:55.528843 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 18:38:55.528900 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:38:55.530102 6474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.521140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:40923 (local address 127.18.125.131:47839)
0806 18:38:55.521840 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:55.521885 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:55.523263 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:55.523929 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:55.523937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:55.524030 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:55.524653 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.524668 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.525575 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.525577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.526623 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.526636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.527277 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.527293 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.527457 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.528242 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:55.528243 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:55.528296 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:55.528371 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:55.528675 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:55.528685 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:55.528690 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:38:55.528842 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 18:38:55.529188 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":136,"threads_started":1}
I20260806 18:38:55.531424 6472 heartbeater.cc:347] Connected to a master server at 127.18.125.190:40923
I20260806 18:38:55.531708 6472 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:55.532375 6472 heartbeater.cc:511] Master 127.18.125.190:40923 requested a full tablet report, sending...
I20260806 18:38:55.533525 6001 ts_manager.cc:194] Registered new tserver with Master: d9d6255684e34a30bb6ad220cb7a2ace (127.18.125.131:33929)
I20260806 18:38:55.534139 6001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.131:47839
I20260806 18:38:55.542245 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:55.548785 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.544516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56210 (local address 127.18.125.190:40923)
0806 18:38:55.544727 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:55.544732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:55.544901 (+ 169us) server_negotiation.cc:407] Connection header received
0806 18:38:55.545073 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:55.545076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:55.545129 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:55.545221 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:55.545522 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.546056 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.546935 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.547160 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.547269 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.548233 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:55.548237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:55.548240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:55.548253 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:55.548281 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:55.548291 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:55.548424 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:55.548574 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 18:38:55.548617 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 18:38:55.552304 6001 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:55.555440 6001 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:55.573092 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.566363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:33929 (local address 127.0.0.1:36766)
0806 18:38:55.567690 (+ 1327us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:55.567721 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:55.567824 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:55.568771 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:55.568775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:55.568797 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:55.568898 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.568905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.570183 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.570189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.571196 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.571206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.572438 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.572443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.572561 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.572624 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 18:38:55.572758 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 18:38:55.572894 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1254,"mutex_wait_us":48,"thread_start_us":123,"threads_started":1}
I20260806 18:38:55.573472 6487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.565780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50474 (local address 127.18.125.129:44149)
0806 18:38:55.566386 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:55.566400 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:55.566740 (+ 340us) server_negotiation.cc:407] Connection header received
0806 18:38:55.566857 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:55.566863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:55.567105 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:55.567281 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:55.567922 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.569390 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.571480 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.572303 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.572435 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.572645 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:55.573165 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 18:38:55.573297 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":152,"threads_started":1}
I20260806 18:38:55.573092 6485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.565459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44149 (local address 127.0.0.1:50474)
0806 18:38:55.566428 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:55.566481 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:55.566644 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:55.567484 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:55.567495 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:55.567527 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:55.567769 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.567778 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.569581 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.569584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.571327 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.571340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.572449 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.572452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.572726 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.572760 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 18:38:55.572837 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:38:55.572894 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"mutex_wait_us":277,"spinlock_wait_cycles":7296,"thread_start_us":135,"threads_started":1}
I20260806 18:38:55.573827 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.567235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36766 (local address 127.18.125.131:33929)
0806 18:38:55.568094 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:55.568105 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:55.568148 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:38:55.568311 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:55.568316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:55.568467 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:55.568803 (+ 336us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:55.569016 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.570035 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.571349 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.572287 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.572437 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.572645 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:55.572810 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 18:38:55.572930 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":128,"threads_started":1}
I20260806 18:38:55.574187 6483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.565459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:33267 (local address 127.0.0.1:50398)
0806 18:38:55.566395 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:55.566465 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:55.566645 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:55.567579 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:55.567583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:55.567611 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:55.567790 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.567800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.569721 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.569724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.571348 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.571354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.573743 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.573747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.573854 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.573884 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 18:38:55.573961 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:38:55.574044 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":283,"spinlock_wait_cycles":7296,"thread_start_us":126,"threads_started":1}
I20260806 18:38:55.574935 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.565609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50398 (local address 127.18.125.130:33267)
0806 18:38:55.566628 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:55.566641 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:55.566673 (+ 32us) server_negotiation.cc:407] Connection header received
0806 18:38:55.567203 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:55.567211 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:55.567421 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:55.567591 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:55.568248 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.569588 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.572750 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.573584 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.574422 (+ 838us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.574595 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:55.574651 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 18:38:55.574757 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":901,"thread_start_us":135,"threads_started":1}
I20260806 18:38:55.576853 6402 tablet_service.cc:1467] Processing CreateTablet for tablet 95670d33ae4741dd8753b0e673436c29 (DEFAULT_TABLE table=test-table [id=c21c901019b240d7b03116c00ac884e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:55.576874 6131 tablet_service.cc:1467] Processing CreateTablet for tablet 95670d33ae4741dd8753b0e673436c29 (DEFAULT_TABLE table=test-table [id=c21c901019b240d7b03116c00ac884e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:55.577466 6267 tablet_service.cc:1467] Processing CreateTablet for tablet 95670d33ae4741dd8753b0e673436c29 (DEFAULT_TABLE table=test-table [id=c21c901019b240d7b03116c00ac884e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:55.577991 6131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95670d33ae4741dd8753b0e673436c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:55.577991 6402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95670d33ae4741dd8753b0e673436c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:55.578338 6267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95670d33ae4741dd8753b0e673436c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:55.585248 6490 tablet_bootstrap.cc:492] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe: Bootstrap starting.
I20260806 18:38:55.586400 6491 tablet_bootstrap.cc:492] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6: Bootstrap starting.
I20260806 18:38:55.587575 6490 tablet_bootstrap.cc:654] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:55.588057 6491 tablet_bootstrap.cc:654] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:55.588403 6490 log.cc:824] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:55.588496 6492 tablet_bootstrap.cc:492] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace: Bootstrap starting.
I20260806 18:38:55.588914 6491 log.cc:824] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:55.590580 6492 tablet_bootstrap.cc:654] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:55.591455 6492 log.cc:824] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:55.591663 6491 tablet_bootstrap.cc:492] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6: No bootstrap required, opened a new log
I20260806 18:38:55.591878 6491 ts_tablet_manager.cc:1397] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 18:38:55.593405 6492 tablet_bootstrap.cc:492] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace: No bootstrap required, opened a new log
I20260806 18:38:55.593614 6492 ts_tablet_manager.cc:1397] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:38:55.593837 6490 tablet_bootstrap.cc:492] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe: No bootstrap required, opened a new log
I20260806 18:38:55.594043 6490 ts_tablet_manager.cc:1397] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260806 18:38:55.596099 6491 raft_consensus.cc:348] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eb35b851854ef787877d4c85b852a6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33267 } } peers { permanent_uuid: "30d20ab440a548e4b439aeda253973fe" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44149 } } peers { permanent_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33929 } }
I20260806 18:38:55.596427 6491 raft_consensus.cc:374] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:55.596480 6491 raft_consensus.cc:729] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64eb35b851854ef787877d4c85b852a6, State: Initialized, Role: FOLLOWER
I20260806 18:38:55.596980 6491 consensus_queue.cc:260] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eb35b851854ef787877d4c85b852a6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33267 } } peers { permanent_uuid: "30d20ab440a548e4b439aeda253973fe" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44149 } } peers { permanent_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33929 } }
I20260806 18:38:55.597369 6492 raft_consensus.cc:348] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eb35b851854ef787877d4c85b852a6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33267 } } peers { permanent_uuid: "30d20ab440a548e4b439aeda253973fe" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44149 } } peers { permanent_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33929 } }
I20260806 18:38:55.597615 6492 raft_consensus.cc:374] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:55.597662 6492 raft_consensus.cc:729] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9d6255684e34a30bb6ad220cb7a2ace, State: Initialized, Role: FOLLOWER
I20260806 18:38:55.597707 6335 heartbeater.cc:503] Master 127.18.125.190:40923 was elected leader, sending a full tablet report...
I20260806 18:38:55.598057 6492 consensus_queue.cc:260] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eb35b851854ef787877d4c85b852a6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33267 } } peers { permanent_uuid: "30d20ab440a548e4b439aeda253973fe" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44149 } } peers { permanent_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33929 } }
I20260806 18:38:55.598044 6491 ts_tablet_manager.cc:1428] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260806 18:38:55.598189 6490 raft_consensus.cc:348] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eb35b851854ef787877d4c85b852a6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33267 } } peers { permanent_uuid: "30d20ab440a548e4b439aeda253973fe" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44149 } } peers { permanent_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33929 } }
I20260806 18:38:55.598423 6490 raft_consensus.cc:374] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:55.598474 6490 raft_consensus.cc:729] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d20ab440a548e4b439aeda253973fe, State: Initialized, Role: FOLLOWER
I20260806 18:38:55.598876 6490 consensus_queue.cc:260] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eb35b851854ef787877d4c85b852a6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33267 } } peers { permanent_uuid: "30d20ab440a548e4b439aeda253973fe" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44149 } } peers { permanent_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33929 } }
I20260806 18:38:55.599076 6492 ts_tablet_manager.cc:1428] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:38:55.599448 6472 heartbeater.cc:503] Master 127.18.125.190:40923 was elected leader, sending a full tablet report...
I20260806 18:38:55.599787 6200 heartbeater.cc:503] Master 127.18.125.190:40923 was elected leader, sending a full tablet report...
I20260806 18:38:55.599992 6490 ts_tablet_manager.cc:1428] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260806 18:38:55.609701 6201 tablet.cc:2369] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:55.648401 6496 raft_consensus.cc:482] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:55.648641 6496 raft_consensus.cc:504] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eb35b851854ef787877d4c85b852a6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33267 } } peers { permanent_uuid: "30d20ab440a548e4b439aeda253973fe" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44149 } } peers { permanent_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33929 } }
I20260806 18:38:55.650122 6496 leader_election.cc:288] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d20ab440a548e4b439aeda253973fe (127.18.125.129:44149), d9d6255684e34a30bb6ad220cb7a2ace (127.18.125.131:33929)
I20260806 18:38:55.654405 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.650583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:53567 (local address 127.18.125.131:33929)
0806 18:38:55.650773 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:55.650781 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:55.650802 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:38:55.650883 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:55.650885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:55.650934 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:55.651054 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:55.651398 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.652006 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.653358 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.653772 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.653916 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.654014 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:55.654203 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 18:38:55.654283 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 18:38:55.654405 6337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.650429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:33929 (local address 127.18.125.130:53567)
0806 18:38:55.650582 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:55.650603 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:55.650731 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:55.651072 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:55.651075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:55.651097 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:55.651254 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.651262 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.652184 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.652189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.653190 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.653202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.653916 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.653920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.654048 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.654079 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:38:55.654170 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 18:38:55.654256 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 18:38:55.655071 6487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.650566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:48745 (local address 127.18.125.129:44149)
0806 18:38:55.650716 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:55.650722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:55.651178 (+ 456us) server_negotiation.cc:407] Connection header received
0806 18:38:55.651353 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:55.651356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:55.651414 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:55.651554 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:55.651877 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.652516 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.653892 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.654375 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.654542 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.654602 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:55.654848 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 18:38:55.654939 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:38:55.655110 6499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.650436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44149 (local address 127.18.125.130:48745)
0806 18:38:55.651059 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:55.651072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:55.651216 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:55.651567 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:55.651570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:55.651588 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:55.651740 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.651747 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.652645 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.652650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.653712 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:55.653726 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.654552 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.654556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:55.654684 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.654713 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:38:55.654807 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:38:55.654895 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":215,"thread_start_us":177,"threads_started":1}
I20260806 18:38:55.655387 6423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95670d33ae4741dd8753b0e673436c29" candidate_uuid: "64eb35b851854ef787877d4c85b852a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" is_pre_election: true
I20260806 18:38:55.655833 6423 raft_consensus.cc:2454] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64eb35b851854ef787877d4c85b852a6 in term 0.
I20260806 18:38:55.655905 6151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95670d33ae4741dd8753b0e673436c29" candidate_uuid: "64eb35b851854ef787877d4c85b852a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d20ab440a548e4b439aeda253973fe" is_pre_election: true
I20260806 18:38:55.656262 6151 raft_consensus.cc:2454] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64eb35b851854ef787877d4c85b852a6 in term 0.
I20260806 18:38:55.656922 6218 leader_election.cc:302] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 30d20ab440a548e4b439aeda253973fe, 64eb35b851854ef787877d4c85b852a6, d9d6255684e34a30bb6ad220cb7a2ace; no voters:
I20260806 18:38:55.657285 6496 raft_consensus.cc:2781] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:55.657459 6496 raft_consensus.cc:482] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:55.657536 6496 raft_consensus.cc:3037] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:55.658805 6496 raft_consensus.cc:504] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eb35b851854ef787877d4c85b852a6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33267 } } peers { permanent_uuid: "30d20ab440a548e4b439aeda253973fe" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44149 } } peers { permanent_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33929 } }
I20260806 18:38:55.659276 6496 leader_election.cc:288] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [CANDIDATE]: Term 1 election: Requested vote from peers 30d20ab440a548e4b439aeda253973fe (127.18.125.129:44149), d9d6255684e34a30bb6ad220cb7a2ace (127.18.125.131:33929)
I20260806 18:38:55.659776 6423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95670d33ae4741dd8753b0e673436c29" candidate_uuid: "64eb35b851854ef787877d4c85b852a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9d6255684e34a30bb6ad220cb7a2ace"
I20260806 18:38:55.659807 6151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95670d33ae4741dd8753b0e673436c29" candidate_uuid: "64eb35b851854ef787877d4c85b852a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d20ab440a548e4b439aeda253973fe"
I20260806 18:38:55.659906 6423 raft_consensus.cc:3037] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:55.659998 6151 raft_consensus.cc:3037] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:55.661351 6423 raft_consensus.cc:2454] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64eb35b851854ef787877d4c85b852a6 in term 1.
I20260806 18:38:55.661445 6151 raft_consensus.cc:2454] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64eb35b851854ef787877d4c85b852a6 in term 1.
I20260806 18:38:55.661777 6221 leader_election.cc:302] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64eb35b851854ef787877d4c85b852a6, d9d6255684e34a30bb6ad220cb7a2ace; no voters:
I20260806 18:38:55.661996 6496 raft_consensus.cc:2781] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:55.662245 6496 raft_consensus.cc:686] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 1 LEADER]: Becoming Leader. State: Replica: 64eb35b851854ef787877d4c85b852a6, State: Running, Role: LEADER
I20260806 18:38:55.662691 6496 consensus_queue.cc:237] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eb35b851854ef787877d4c85b852a6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33267 } } peers { permanent_uuid: "30d20ab440a548e4b439aeda253973fe" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44149 } } peers { permanent_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33929 } }
I20260806 18:38:55.666690 6001 catalog_manager.cc:5495] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64eb35b851854ef787877d4c85b852a6 (127.18.125.130). New cstate: current_term: 1 leader_uuid: "64eb35b851854ef787877d4c85b852a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eb35b851854ef787877d4c85b852a6" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 33267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d20ab440a548e4b439aeda253973fe" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33929 } health_report { overall_health: UNKNOWN } } }
I20260806 18:38:55.698911 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.694913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50404 (local address 127.18.125.130:33267)
0806 18:38:55.695063 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:55.695067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:55.695081 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:38:55.695156 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:55.695158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:55.695216 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:55.695305 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:55.695564 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.696073 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.697085 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:55.697320 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:55.697404 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:55.697727 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:55.698205 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:55.698328 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:55.698758 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 18:38:55.698796 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:38:55.706228 6423 raft_consensus.cc:1261] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace [term 1 FOLLOWER]: Refusing update from remote peer 64eb35b851854ef787877d4c85b852a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:55.706228 6151 raft_consensus.cc:1261] T 95670d33ae4741dd8753b0e673436c29 P 30d20ab440a548e4b439aeda253973fe [term 1 FOLLOWER]: Refusing update from remote peer 64eb35b851854ef787877d4c85b852a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:55.707079 6500 consensus_queue.cc:1035] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d20ab440a548e4b439aeda253973fe" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:55.707510 6496 consensus_queue.cc:1035] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9d6255684e34a30bb6ad220cb7a2ace" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:55.717048 6505 mvcc.cc:205] Tried to move back new op lower bound from 7315626130243735552 to 7315626130077593600. Current Snapshot: MvccSnapshot[applied={T|T < 7315626130243735552}]
I20260806 18:38:55.726809 5998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:55.727034 5998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:55.730111 5998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71a26f9f21e142b19c305c019ccdb7d8: Sending DeleteTablet for 3 replicas of tablet 95670d33ae4741dd8753b0e673436c29
I20260806 18:38:55.731205 6131 tablet_service.cc:1514] Processing DeleteTablet for tablet 95670d33ae4741dd8753b0e673436c29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:55 UTC) from {username='slave'} at 127.0.0.1:50474
I20260806 18:38:55.731391 6402 tablet_service.cc:1514] Processing DeleteTablet for tablet 95670d33ae4741dd8753b0e673436c29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:55 UTC) from {username='slave'} at 127.0.0.1:36766
I20260806 18:38:55.731885 6267 tablet_service.cc:1514] Processing DeleteTablet for tablet 95670d33ae4741dd8753b0e673436c29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:55 UTC) from {username='slave'} at 127.0.0.1:50398
I20260806 18:38:55.732281 6515 tablet_replica.cc:330] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace: stopping tablet replica
I20260806 18:38:55.732322 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 6066
I20260806 18:38:55.732761 6515 raft_consensus.cc:2229] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:55.733512 6516 tablet_replica.cc:330] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6: stopping tablet replica
I20260806 18:38:55.733541 6515 raft_consensus.cc:2258] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:55.733839 6516 raft_consensus.cc:2229] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:38:55.734880 6516 raft_consensus.cc:2258] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:55.735771 6515 ts_tablet_manager.cc:1905] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:55.736474 6516 ts_tablet_manager.cc:1905] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:55.740032 6515 ts_tablet_manager.cc:1918] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:55.740233 6515 log.cc:1196] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/95670d33ae4741dd8753b0e673436c29
I20260806 18:38:55.740672 6515 ts_tablet_manager.cc:1939] T 95670d33ae4741dd8753b0e673436c29 P d9d6255684e34a30bb6ad220cb7a2ace: Deleting consensus metadata
I20260806 18:38:55.741948 6516 ts_tablet_manager.cc:1918] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:55.742343 6516 log.cc:1196] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/95670d33ae4741dd8753b0e673436c29
I20260806 18:38:55.742641 5990 catalog_manager.cc:4841] TS d9d6255684e34a30bb6ad220cb7a2ace (127.18.125.131:33929): tablet 95670d33ae4741dd8753b0e673436c29 (table test-table [id=c21c901019b240d7b03116c00ac884e5]) successfully deleted
I20260806 18:38:55.742957 6516 ts_tablet_manager.cc:1939] T 95670d33ae4741dd8753b0e673436c29 P 64eb35b851854ef787877d4c85b852a6: Deleting consensus metadata
I20260806 18:38:55.744434 5990 catalog_manager.cc:4841] TS 64eb35b851854ef787877d4c85b852a6 (127.18.125.130:33267): tablet 95670d33ae4741dd8753b0e673436c29 (table test-table [id=c21c901019b240d7b03116c00ac884e5]) successfully deleted
W20260806 18:38:55.746387 5987 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:44149 (error 108)
I20260806 18:38:55.747418 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 6203
I20260806 18:38:55.747759 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:55.747436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44149 (local address 127.0.0.1:50486)
0806 18:38:55.747578 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:55.747654 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:44149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 18:38:55.749735 5987 catalog_manager.cc:4584] TS 30d20ab440a548e4b439aeda253973fe (127.18.125.129:44149): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95670d33ae4741dd8753b0e673436c29: Network error: Client connection negotiation failed: client connection to 127.18.125.129:44149: connect: Connection refused (error 111)
I20260806 18:38:55.755935 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 6338
I20260806 18:38:55.765959 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 5972
2026-08-06T18:38:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1138 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:55 dist-test-slave-8w9v krb5kdc[6521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:55 dist-test-slave-8w9v krb5kdc[6521](info): set up 2 sockets
Aug 06 18:38:55 dist-test-slave-8w9v krb5kdc[6521](info): commencing operation
krb5kdc: starting...
W20260806 18:38:56.869328 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:38:56 dist-test-slave-8w9v krb5kdc[6521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:38:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:38:57.017462 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:33683
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:33683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:57.136380 6535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:57.136678 6535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:57.136721 6535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:57.140820 6535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:57.140895 6535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:57.140915 6535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:57.140935 6535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:57.140951 6535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:57.146502 6535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:33683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:33683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:57.147627 6535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:57.148730 6535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:57.157968 6543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:57.158046 6535 server_base.cc:1034] running on GCE node
W20260806 18:38:57.157998 6541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:57.157998 6540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:57.158964 6535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:57.160279 6535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:57.161458 6535 hybrid_clock.cc:648] HybridClock initialized: now 1786041537161439 us; error 34 us; skew 500 ppm
Aug 06 18:38:57 dist-test-slave-8w9v krb5kdc[6521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041537, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:57.164675 6535 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:38:57.165834 6535 webserver.cc:460] Webserver started at http://127.18.125.190:39295/ using document root <none> and password file <none>
I20260806 18:38:57.166530 6535 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:57.166601 6535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:57.166872 6535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:57.169248 6535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "95b17761efed42b3ae955667095ddb6f"
format_stamp: "Formatted at 2026-08-06 18:38:57 on dist-test-slave-8w9v"
server_key: "b674e1bc0c64b5f8f0cf72b0d3920d00"
server_key_iv: "0047922b7742b51fa5a94130e718c5b0"
server_key_version: "encryptionkey@0"
I20260806 18:38:57.169905 6535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "95b17761efed42b3ae955667095ddb6f"
format_stamp: "Formatted at 2026-08-06 18:38:57 on dist-test-slave-8w9v"
server_key: "b674e1bc0c64b5f8f0cf72b0d3920d00"
server_key_iv: "0047922b7742b51fa5a94130e718c5b0"
server_key_version: "encryptionkey@0"
I20260806 18:38:57.174467 6535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:38:57.177214 6550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:57.178601 6535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:57.178730 6535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "95b17761efed42b3ae955667095ddb6f"
format_stamp: "Formatted at 2026-08-06 18:38:57 on dist-test-slave-8w9v"
server_key: "b674e1bc0c64b5f8f0cf72b0d3920d00"
server_key_iv: "0047922b7742b51fa5a94130e718c5b0"
server_key_version: "encryptionkey@0"
I20260806 18:38:57.178835 6535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:57.199671 6535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:57.202849 6535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:57.203019 6535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:57.210969 6535 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:33683
I20260806 18:38:57.210997 6604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:33683 every 8 connection(s)
I20260806 18:38:57.212056 6535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:57.215080 6605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:57.215554 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 6535
I20260806 18:38:57.215708 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:38:57.216055 18934 external_mini_cluster.cc:1428] Setting key 9c5ecb96264e9fd2dae5589af9b8272a
I20260806 18:38:57.223096 6605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f: Bootstrap starting.
Aug 06 18:38:57 dist-test-slave-8w9v krb5kdc[6521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:57.226127 6605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:57.226989 6605 log.cc:824] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:57.231719 6605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f: No bootstrap required, opened a new log
I20260806 18:38:57.233085 6608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:57.217815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.18.125.190:33683)
0806 18:38:57.218363 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:57.218380 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:57.218417 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:38:57.219398 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:57.219419 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:57.219788 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:57.220185 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:57.221070 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:57.222067 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:57.222983 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:57.223538 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:57.223718 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:57.226891 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:57.226896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:57.226913 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:57.226946 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:57.229452 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:57.230071 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:57.230078 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:57.230084 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:57.230231 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:57.230580 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:57.230585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:57.230587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:57.231069 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:57.231322 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:57.231664 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 18:38:57.232070 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":159,"threads_started":1}
I20260806 18:38:57.237564 6605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b17761efed42b3ae955667095ddb6f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33683 } }
I20260806 18:38:57.237906 6605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:57.238018 6605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b17761efed42b3ae955667095ddb6f, State: Initialized, Role: FOLLOWER
I20260806 18:38:57.238588 6605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b17761efed42b3ae955667095ddb6f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33683 } }
I20260806 18:38:57.238793 6605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:57.238873 6605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:57.238996 6605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:57.240304 6605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b17761efed42b3ae955667095ddb6f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33683 } }
I20260806 18:38:57.240895 6605 leader_election.cc:302] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95b17761efed42b3ae955667095ddb6f; no voters:
I20260806 18:38:57.241555 6605 leader_election.cc:288] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:57.241663 6610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:57.241916 6610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [term 1 LEADER]: Becoming Leader. State: Replica: 95b17761efed42b3ae955667095ddb6f, State: Running, Role: LEADER
I20260806 18:38:57.242405 6610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b17761efed42b3ae955667095ddb6f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33683 } }
I20260806 18:38:57.243335 6605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:57.244612 6612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95b17761efed42b3ae955667095ddb6f. Latest consensus state: current_term: 1 leader_uuid: "95b17761efed42b3ae955667095ddb6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b17761efed42b3ae955667095ddb6f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33683 } } }
I20260806 18:38:57.244791 6612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:57.245193 6611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95b17761efed42b3ae955667095ddb6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b17761efed42b3ae955667095ddb6f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33683 } } }
I20260806 18:38:57.245312 6611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:57.245764 6618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:57.250058 6618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:57.257537 6618 catalog_manager.cc:1347] Generated new cluster ID: 5c84fb9fb1554ae0ab8112bee2cc696b
I20260806 18:38:57.257647 6618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:57.279870 6618 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:57.281405 6618 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:57.287238 6618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f: Generated new TSK 0
I20260806 18:38:57.288281 6618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.125.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
Entry for principal kudu/127.18.125.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab.
I20260806 18:38:57.332597 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:33683
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:57.456557 6632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:57.456873 6632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:57.456923 6632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:57.461300 6632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:57.461402 6632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:57.461470 6632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:38:57.467422 6632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.129.keytab
--principal=kudu/127.18.125.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:57.468677 6632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:57.469805 6632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:57.481186 6638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:57.481175 6637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:57.481176 6640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:57.481280 6632 server_base.cc:1034] running on GCE node
I20260806 18:38:57.481885 6632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:57.482827 6632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:57.484014 6632 hybrid_clock.cc:648] HybridClock initialized: now 1786041537483998 us; error 28 us; skew 500 ppm
Aug 06 18:38:57 dist-test-slave-8w9v krb5kdc[6521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041537, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:57.487113 6632 init.cc:377] Logged in from keytab as kudu/127.18.125.129@KRBTEST.COM (short username kudu)
I20260806 18:38:57.488059 6632 webserver.cc:460] Webserver started at http://127.18.125.129:39889/ using document root <none> and password file <none>
I20260806 18:38:57.488729 6632 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:57.488799 6632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:57.489032 6632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:57.491243 6632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b"
format_stamp: "Formatted at 2026-08-06 18:38:57 on dist-test-slave-8w9v"
server_key: "05012953c50b2f0d916a7e435e3258e0"
server_key_iv: "18c2cd1f009e08fb404a3e233beff6b1"
server_key_version: "encryptionkey@0"
I20260806 18:38:57.491882 6632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b"
format_stamp: "Formatted at 2026-08-06 18:38:57 on dist-test-slave-8w9v"
server_key: "05012953c50b2f0d916a7e435e3258e0"
server_key_iv: "18c2cd1f009e08fb404a3e233beff6b1"
server_key_version: "encryptionkey@0"
I20260806 18:38:57.495972 6632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:38:57.499115 6648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:57.500658 6632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:57.500802 6632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b"
format_stamp: "Formatted at 2026-08-06 18:38:57 on dist-test-slave-8w9v"
server_key: "05012953c50b2f0d916a7e435e3258e0"
server_key_iv: "18c2cd1f009e08fb404a3e233beff6b1"
server_key_version: "encryptionkey@0"
I20260806 18:38:57.500931 6632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:57.515667 6632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:57.519279 6632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:57.519507 6632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:57.520231 6632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:57.521194 6632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:57.521245 6632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:57.521296 6632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:57.521310 6632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:57.530712 6632 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:44429
I20260806 18:38:57.530786 6765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:44429 every 8 connection(s)
I20260806 18:38:57.531697 6632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:57.541124 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 6632
I20260806 18:38:57.541316 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:57.541636 18934 external_mini_cluster.cc:1428] Setting key 2f2b0379ef210527bb405469741872ca
Aug 06 18:38:57 dist-test-slave-8w9v krb5kdc[6521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041537, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.129@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:57.547513 6608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:57.533789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:58595 (local address 127.18.125.190:33683)
0806 18:38:57.533962 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:57.533968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:57.534927 (+ 959us) server_negotiation.cc:407] Connection header received
0806 18:38:57.536389 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:57.536392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:57.536444 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:57.536527 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:57.537889 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:57.538461 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:57.539722 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:57.540027 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:57.540179 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:57.543179 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:57.543182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:57.543185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:57.543221 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:57.545104 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:57.545716 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:57.545720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:57.545722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:57.545846 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:57.546216 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:57.546221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:57.546222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:57.546441 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:57.546561 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:57.547247 (+ 686us) server_negotiation.cc:299] Negotiation successful
0806 18:38:57.547384 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 18:38:57.548914 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:57.534233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:33683 (local address 127.18.125.129:58595)
0806 18:38:57.534753 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:57.534801 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:57.536154 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:57.536688 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:57.536695 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:57.537094 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:57.537695 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:57.537711 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:57.538614 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:57.538617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:57.539567 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:57.539578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:57.540315 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:57.540319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:57.540478 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:57.541184 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:57.541203 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:57.542974 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:57.545266 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:57.545275 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:57.545298 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:57.545597 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:57.545943 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:57.545946 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:57.545948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:57.546111 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:57.546561 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:57.546570 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:57.546946 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 18:38:57.547187 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 18:38:57.547610 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":112,"threads_started":1}
I20260806 18:38:57.550521 6766 heartbeater.cc:347] Connected to a master server at 127.18.125.190:33683
I20260806 18:38:57.550812 6766 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:57.551355 6766 heartbeater.cc:511] Master 127.18.125.190:33683 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.125.130@KRBTEST.COM; defaulting to no policy
I20260806 18:38:57.553189 6568 ts_manager.cc:194] Registered new tserver with Master: 73a7462c7b9f4581b2c4e2f2d0d2f29b (127.18.125.129:44429)
I20260806 18:38:57.554488 6568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.129@KRBTEST.COM'} at 127.18.125.129:58595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
Entry for principal kudu/127.18.125.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab.
I20260806 18:38:57.576416 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:33683
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:57.697324 6771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:57.697700 6771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:57.697793 6771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:57.702286 6771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:57.702569 6771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:57.702713 6771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:38:57.709328 6771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.130.keytab
--principal=kudu/127.18.125.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:57.710610 6771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:57.711791 6771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:57.724952 6776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:57.724968 6777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:57.724968 6779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:57.725193 6771 server_base.cc:1034] running on GCE node
I20260806 18:38:57.725766 6771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:57.726521 6771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:57.727733 6771 hybrid_clock.cc:648] HybridClock initialized: now 1786041537727714 us; error 30 us; skew 500 ppm
Aug 06 18:38:57 dist-test-slave-8w9v krb5kdc[6521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041537, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:57.730768 6771 init.cc:377] Logged in from keytab as kudu/127.18.125.130@KRBTEST.COM (short username kudu)
I20260806 18:38:57.731827 6771 webserver.cc:460] Webserver started at http://127.18.125.130:36715/ using document root <none> and password file <none>
I20260806 18:38:57.732565 6771 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:57.732668 6771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:57.732933 6771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:57.735177 6771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "43b5faf26d4e4a87bcc115a19bedbdec"
format_stamp: "Formatted at 2026-08-06 18:38:57 on dist-test-slave-8w9v"
server_key: "550fda9bc4fada484906dfaa922f8613"
server_key_iv: "7cac418d00212fefb4c18ba22c562aae"
server_key_version: "encryptionkey@0"
I20260806 18:38:57.735814 6771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "43b5faf26d4e4a87bcc115a19bedbdec"
format_stamp: "Formatted at 2026-08-06 18:38:57 on dist-test-slave-8w9v"
server_key: "550fda9bc4fada484906dfaa922f8613"
server_key_iv: "7cac418d00212fefb4c18ba22c562aae"
server_key_version: "encryptionkey@0"
I20260806 18:38:57.740268 6771 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:38:57.743089 6787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:57.744925 6771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:57.745083 6771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "43b5faf26d4e4a87bcc115a19bedbdec"
format_stamp: "Formatted at 2026-08-06 18:38:57 on dist-test-slave-8w9v"
server_key: "550fda9bc4fada484906dfaa922f8613"
server_key_iv: "7cac418d00212fefb4c18ba22c562aae"
server_key_version: "encryptionkey@0"
I20260806 18:38:57.745262 6771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:57.769897 6771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:57.773648 6771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:57.773886 6771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:57.774627 6771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:57.775570 6771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:57.775646 6771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:57.775722 6771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:57.775767 6771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:57.785657 6771 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:35867
I20260806 18:38:57.785722 6906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:35867 every 8 connection(s)
I20260806 18:38:57.786639 6771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:57.794764 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 6771
I20260806 18:38:57.794921 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:57.795243 18934 external_mini_cluster.cc:1428] Setting key 7f25f0b1eed0f062632cf580b805ac39
Aug 06 18:38:57 dist-test-slave-8w9v krb5kdc[6521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041537, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.130@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:57.802510 6608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:57.788647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:58051 (local address 127.18.125.190:33683)
0806 18:38:57.788840 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:57.788844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:57.789741 (+ 897us) server_negotiation.cc:407] Connection header received
0806 18:38:57.791234 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:57.791238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:57.791303 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:57.791420 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:57.792775 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:57.793331 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:57.794633 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:57.794930 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:57.795068 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:57.797932 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:57.797938 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:57.797944 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:57.797979 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:57.799595 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:57.800452 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:57.800455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:57.800457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:57.800538 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:57.800999 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:57.801002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:57.801003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:57.801210 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:57.801290 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:57.802234 (+ 944us) server_negotiation.cc:299] Negotiation successful
0806 18:38:57.802380 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 18:38:57.803769 6909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:57.789035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:33683 (local address 127.18.125.130:58051)
0806 18:38:57.789543 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:57.789593 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:57.790983 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:57.791612 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:57.791621 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:57.792015 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:57.792601 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:57.792618 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:57.793484 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:57.793493 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:57.794472 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:57.794486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:57.795131 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:57.795135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:57.795316 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:57.796031 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:57.796048 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:57.797738 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:57.799734 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:57.799744 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:57.799769 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:57.800325 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:57.800648 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:57.800654 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:57.800659 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:57.800881 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:57.801330 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:57.801341 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:57.801748 (+ 407us) client_negotiation.cc:769] Sending connection context
0806 18:38:57.802024 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 18:38:57.802472 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":104,"threads_started":1}
I20260806 18:38:57.805161 6907 heartbeater.cc:347] Connected to a master server at 127.18.125.190:33683
I20260806 18:38:57.805473 6907 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:57.806072 6907 heartbeater.cc:511] Master 127.18.125.190:33683 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.125.131@KRBTEST.COM; defaulting to no policy
I20260806 18:38:57.807248 6568 ts_manager.cc:194] Registered new tserver with Master: 43b5faf26d4e4a87bcc115a19bedbdec (127.18.125.130:35867)
I20260806 18:38:57.807878 6568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.130@KRBTEST.COM'} at 127.18.125.130:58051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
Entry for principal kudu/127.18.125.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab.
I20260806 18:38:57.830557 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:33683
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:38:57.972077 6913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:57.972488 6913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:57.972539 6913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:57.978269 6913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:57.978385 6913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:57.978461 6913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:38:57.984930 6913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.131.keytab
--principal=kudu/127.18.125.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:33683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:57.986229 6913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:57.987373 6913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:57.999182 6919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:57.999241 6918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:57.999361 6921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:58.000260 6913 server_base.cc:1034] running on GCE node
I20260806 18:38:58.000746 6913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:58.001468 6913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:58.002691 6913 hybrid_clock.cc:648] HybridClock initialized: now 1786041538002676 us; error 33 us; skew 500 ppm
Aug 06 18:38:58 dist-test-slave-8w9v krb5kdc[6521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041538, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:38:58.006851 6913 init.cc:377] Logged in from keytab as kudu/127.18.125.131@KRBTEST.COM (short username kudu)
I20260806 18:38:58.007992 6913 webserver.cc:460] Webserver started at http://127.18.125.131:44571/ using document root <none> and password file <none>
I20260806 18:38:58.008735 6913 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:58.008860 6913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:58.009138 6913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:58.011727 6913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "477944b8c22d4e01836baf0bbba0da78"
format_stamp: "Formatted at 2026-08-06 18:38:58 on dist-test-slave-8w9v"
server_key: "4a312e60165c87eb106b99956441088c"
server_key_iv: "eaf7a37e06a08cde2fa1a42f435b9135"
server_key_version: "encryptionkey@0"
I20260806 18:38:58.012463 6913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "477944b8c22d4e01836baf0bbba0da78"
format_stamp: "Formatted at 2026-08-06 18:38:58 on dist-test-slave-8w9v"
server_key: "4a312e60165c87eb106b99956441088c"
server_key_iv: "eaf7a37e06a08cde2fa1a42f435b9135"
server_key_version: "encryptionkey@0"
I20260806 18:38:58.016892 6913 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:38:58.019757 6928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:58.021350 6913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:58.021514 6913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "477944b8c22d4e01836baf0bbba0da78"
format_stamp: "Formatted at 2026-08-06 18:38:58 on dist-test-slave-8w9v"
server_key: "4a312e60165c87eb106b99956441088c"
server_key_iv: "eaf7a37e06a08cde2fa1a42f435b9135"
server_key_version: "encryptionkey@0"
I20260806 18:38:58.021672 6913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:58.066268 6913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:58.070101 6913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:58.070351 6913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:58.071084 6913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:58.072163 6913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:58.072243 6913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:58.072322 6913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:58.072376 6913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:58.083865 6913 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:43991
I20260806 18:38:58.083885 7046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:43991 every 8 connection(s)
I20260806 18:38:58.084935 6913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:58.092079 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 6913
I20260806 18:38:58.092316 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:58.092664 18934 external_mini_cluster.cc:1428] Setting key 601b044a3c76adc13a41b3bf4e6b22a6
Aug 06 18:38:58 dist-test-slave-8w9v krb5kdc[6521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041538, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.131@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:38:58.106676 6608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.087352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:35593 (local address 127.18.125.190:33683)
0806 18:38:58.087521 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:58.087526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:58.088969 (+ 1443us) server_negotiation.cc:407] Connection header received
0806 18:38:58.090530 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:58.090535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:58.090609 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:58.090739 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:58.092272 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.092973 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.095285 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.095754 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.095873 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.100436 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:58.100441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:58.100443 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:58.100476 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:58.102857 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:58.104044 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:58.104052 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:58.104053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:58.104252 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:58.104930 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:58.104937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:58.104939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:58.105188 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:58.105286 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:58.106290 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0806 18:38:58.106461 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:38:58.107800 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.087934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:33683 (local address 127.18.125.131:35593)
0806 18:38:58.088606 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:58.088817 (+ 211us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:58.090254 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:58.090962 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:58.090971 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:58.091432 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:58.092011 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.092028 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.093172 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.093178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.095083 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.095102 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.096251 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.096261 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.096579 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.097625 (+ 1046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:38:58.097652 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:58.099882 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:58.103159 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:58.103170 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:58.103194 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:58.103790 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:58.104413 (+ 623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:38:58.104421 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:38:58.104423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:38:58.104733 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:38:58.105346 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:58.105353 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:58.105735 (+ 382us) client_negotiation.cc:769] Sending connection context
0806 18:38:58.106140 (+ 405us) client_negotiation.cc:240] Negotiation successful
0806 18:38:58.106612 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":217,"threads_started":1}
I20260806 18:38:58.109093 7047 heartbeater.cc:347] Connected to a master server at 127.18.125.190:33683
I20260806 18:38:58.109414 7047 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:58.110162 7047 heartbeater.cc:511] Master 127.18.125.190:33683 requested a full tablet report, sending...
I20260806 18:38:58.111797 6568 ts_manager.cc:194] Registered new tserver with Master: 477944b8c22d4e01836baf0bbba0da78 (127.18.125.131:43991)
I20260806 18:38:58.112515 6568 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.125.131@KRBTEST.COM'} at 127.18.125.131:35593
I20260806 18:38:58.120219 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:58.132915 6608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.122799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42666 (local address 127.18.125.190:33683)
0806 18:38:58.122947 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:58.122951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:58.123259 (+ 308us) server_negotiation.cc:407] Connection header received
0806 18:38:58.123401 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:58.123403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:58.123454 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:58.123536 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:58.124873 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.125741 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.126970 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.127247 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.127361 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.129032 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:58.129036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:58.129038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:38:58.129065 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:58.130627 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:58.131153 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:58.131156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:58.131157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:58.131257 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:38:58.131601 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:38:58.131605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:38:58.131606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:38:58.131884 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:58.132307 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:58.132635 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 18:38:58.132777 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:38:58.136699 6568 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:58.139932 6568 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:58.157078 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.150932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52872 (local address 127.18.125.130:35867)
0806 18:38:58.151755 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:58.151792 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:58.151820 (+ 28us) server_negotiation.cc:407] Connection header received
0806 18:38:58.151908 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:58.151914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:58.152187 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:58.152344 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:58.152852 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.153790 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.155387 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.156008 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.156189 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.156352 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:58.156802 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 18:38:58.156920 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":160,"threads_started":1}
I20260806 18:38:58.157074 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.150624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:35867 (local address 127.0.0.1:52872)
0806 18:38:58.151275 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:58.151337 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:58.151477 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:58.152471 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:58.152477 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:58.152506 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:58.152680 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.152691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.153936 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.153939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.155062 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.155097 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.156426 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.156430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.156579 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.156619 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:38:58.156741 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 18:38:58.156873 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":5,"thread_start_us":138,"threads_started":1}
I20260806 18:38:58.157768 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.150631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:43991 (local address 127.0.0.1:37246)
0806 18:38:58.151955 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:58.151968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:58.152085 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:58.152573 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:58.152577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:58.152599 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:58.152762 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.152769 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.154699 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.154703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.155969 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.155982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.157376 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.157379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.157494 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.157514 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:38:58.157559 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 18:38:58.157636 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"mutex_wait_us":661,"thread_start_us":112,"threads_started":1}
I20260806 18:38:58.158103 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.151650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44429 (local address 127.0.0.1:51462)
0806 18:38:58.152665 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:58.152677 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:58.152784 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:58.153214 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:58.153216 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:58.153236 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:58.153317 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.153323 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.154713 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.154716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.156502 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.156512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.157686 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.157689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.157770 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.157801 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:38:58.157883 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:38:58.157972 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"mutex_wait_us":24,"thread_start_us":87,"threads_started":1}
I20260806 18:38:58.158540 7064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.152017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51462 (local address 127.18.125.129:44429)
0806 18:38:58.152485 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:58.152491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:58.152766 (+ 275us) server_negotiation.cc:407] Connection header received
0806 18:38:58.152946 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:58.152951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:58.153080 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:58.153185 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:58.153458 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.154588 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.156636 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.157549 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.157685 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.157805 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:58.158273 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 18:38:58.158384 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":156,"threads_started":1}
I20260806 18:38:58.158810 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.150904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37246 (local address 127.18.125.131:43991)
0806 18:38:58.151441 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:58.151455 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:58.152053 (+ 598us) server_negotiation.cc:407] Connection header received
0806 18:38:58.152235 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:58.152241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:58.152438 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:58.152585 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:58.153319 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.154523 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.156152 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.157069 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.158305 (+ 1236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.158439 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:58.158489 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 18:38:58.158670 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":193,"threads_started":1}
I20260806 18:38:58.161226 6697 tablet_service.cc:1467] Processing CreateTablet for tablet 093517769e6142a7bbec7cfb0ad7250b (DEFAULT_TABLE table=test-table [id=78e9a1931b644faa83f26e973e66ca37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:58.161517 6835 tablet_service.cc:1467] Processing CreateTablet for tablet 093517769e6142a7bbec7cfb0ad7250b (DEFAULT_TABLE table=test-table [id=78e9a1931b644faa83f26e973e66ca37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:58.161860 6978 tablet_service.cc:1467] Processing CreateTablet for tablet 093517769e6142a7bbec7cfb0ad7250b (DEFAULT_TABLE table=test-table [id=78e9a1931b644faa83f26e973e66ca37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:58.162405 6697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 093517769e6142a7bbec7cfb0ad7250b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:58.162649 6835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 093517769e6142a7bbec7cfb0ad7250b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:58.162897 6978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 093517769e6142a7bbec7cfb0ad7250b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:58.169375 7065 tablet_bootstrap.cc:492] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b: Bootstrap starting.
I20260806 18:38:58.171343 7066 tablet_bootstrap.cc:492] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: Bootstrap starting.
I20260806 18:38:58.171455 7067 tablet_bootstrap.cc:492] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: Bootstrap starting.
I20260806 18:38:58.172071 7065 tablet_bootstrap.cc:654] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:58.173100 7065 log.cc:824] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:58.173755 7067 tablet_bootstrap.cc:654] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:58.173934 7066 tablet_bootstrap.cc:654] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:58.174980 7067 log.cc:824] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:58.175071 7066 log.cc:824] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:58.177150 7066 tablet_bootstrap.cc:492] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: No bootstrap required, opened a new log
I20260806 18:38:58.177160 7067 tablet_bootstrap.cc:492] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: No bootstrap required, opened a new log
I20260806 18:38:58.177383 7066 ts_tablet_manager.cc:1397] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:38:58.177382 7067 ts_tablet_manager.cc:1397] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:38:58.180562 7065 tablet_bootstrap.cc:492] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b: No bootstrap required, opened a new log
I20260806 18:38:58.180812 7065 ts_tablet_manager.cc:1397] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260806 18:38:58.181149 7067 raft_consensus.cc:348] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 35867 } } peers { permanent_uuid: "477944b8c22d4e01836baf0bbba0da78" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43991 } } peers { permanent_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44429 } }
I20260806 18:38:58.181419 7067 raft_consensus.cc:374] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:58.181485 7067 raft_consensus.cc:729] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477944b8c22d4e01836baf0bbba0da78, State: Initialized, Role: FOLLOWER
I20260806 18:38:58.181313 7066 raft_consensus.cc:348] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 35867 } } peers { permanent_uuid: "477944b8c22d4e01836baf0bbba0da78" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43991 } } peers { permanent_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44429 } }
I20260806 18:38:58.181583 7066 raft_consensus.cc:374] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:58.181648 7066 raft_consensus.cc:729] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b5faf26d4e4a87bcc115a19bedbdec, State: Initialized, Role: FOLLOWER
I20260806 18:38:58.182027 7067 consensus_queue.cc:260] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 35867 } } peers { permanent_uuid: "477944b8c22d4e01836baf0bbba0da78" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43991 } } peers { permanent_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44429 } }
I20260806 18:38:58.182106 7066 consensus_queue.cc:260] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 35867 } } peers { permanent_uuid: "477944b8c22d4e01836baf0bbba0da78" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43991 } } peers { permanent_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44429 } }
I20260806 18:38:58.182883 7047 heartbeater.cc:503] Master 127.18.125.190:33683 was elected leader, sending a full tablet report...
I20260806 18:38:58.183264 7067 ts_tablet_manager.cc:1428] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:38:58.183295 7066 ts_tablet_manager.cc:1428] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:38:58.183655 6907 heartbeater.cc:503] Master 127.18.125.190:33683 was elected leader, sending a full tablet report...
I20260806 18:38:58.185657 7065 raft_consensus.cc:348] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 35867 } } peers { permanent_uuid: "477944b8c22d4e01836baf0bbba0da78" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43991 } } peers { permanent_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44429 } }
I20260806 18:38:58.186096 7065 raft_consensus.cc:374] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:58.186404 7065 raft_consensus.cc:729] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73a7462c7b9f4581b2c4e2f2d0d2f29b, State: Initialized, Role: FOLLOWER
I20260806 18:38:58.187071 7065 consensus_queue.cc:260] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 35867 } } peers { permanent_uuid: "477944b8c22d4e01836baf0bbba0da78" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43991 } } peers { permanent_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44429 } }
I20260806 18:38:58.188079 6766 heartbeater.cc:503] Master 127.18.125.190:33683 was elected leader, sending a full tablet report...
I20260806 18:38:58.188501 7065 ts_tablet_manager.cc:1428] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b: Time spent starting tablet: real 0.008s user 0.002s sys 0.006s
W20260806 18:38:58.283351 6767 tablet.cc:2369] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:38:58.287954 6908 tablet.cc:2369] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:38:58.336305 7048 tablet.cc:2369] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:58.344002 7072 raft_consensus.cc:482] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:58.344309 7072 raft_consensus.cc:504] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 35867 } } peers { permanent_uuid: "477944b8c22d4e01836baf0bbba0da78" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43991 } } peers { permanent_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44429 } }
I20260806 18:38:58.345638 7072 leader_election.cc:288] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 477944b8c22d4e01836baf0bbba0da78 (127.18.125.131:43991), 73a7462c7b9f4581b2c4e2f2d0d2f29b (127.18.125.129:44429)
I20260806 18:38:58.350344 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.346136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:33513 (local address 127.18.125.131:43991)
0806 18:38:58.346283 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:58.346287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:58.346528 (+ 241us) server_negotiation.cc:407] Connection header received
0806 18:38:58.346719 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:58.346723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:58.346811 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:58.346895 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:58.347317 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.348054 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.349251 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.349722 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.349821 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.349865 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:58.350117 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 18:38:58.350211 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:38:58.350363 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.346014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:43991 (local address 127.18.125.130:33513)
0806 18:38:58.346285 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:58.346303 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:58.346527 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:58.347048 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:58.347052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:58.347070 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:58.347204 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.347211 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.348211 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.348214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.349110 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.349120 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.349855 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.349859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.349968 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.350002 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 18:38:58.350079 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:38:58.350171 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":1,"thread_start_us":112,"threads_started":1}
I20260806 18:38:58.350376 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.346419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44429 (local address 127.18.125.130:49307)
0806 18:38:58.346709 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:58.346723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:58.346851 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:58.347179 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:58.347182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:58.347202 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:58.347359 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.347367 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.348376 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.348378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.349236 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.349247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.349970 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.349972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.350049 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.350102 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 18:38:58.350181 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:38:58.350248 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":72,"threads_started":1}
I20260806 18:38:58.350425 7064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.346589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:49307 (local address 127.18.125.129:44429)
0806 18:38:58.346748 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:58.346752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:58.346830 (+ 78us) server_negotiation.cc:407] Connection header received
0806 18:38:58.346984 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:58.346987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:58.347055 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:58.347163 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:58.347516 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.348264 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.349377 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.349821 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.349956 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.350007 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:58.350219 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 18:38:58.350302 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:38:58.351259 6717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093517769e6142a7bbec7cfb0ad7250b" candidate_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" is_pre_election: true
I20260806 18:38:58.351264 6999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093517769e6142a7bbec7cfb0ad7250b" candidate_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477944b8c22d4e01836baf0bbba0da78" is_pre_election: true
I20260806 18:38:58.351644 6999 raft_consensus.cc:2454] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43b5faf26d4e4a87bcc115a19bedbdec in term 0.
I20260806 18:38:58.351645 6717 raft_consensus.cc:2454] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43b5faf26d4e4a87bcc115a19bedbdec in term 0.
I20260806 18:38:58.352210 6788 leader_election.cc:302] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43b5faf26d4e4a87bcc115a19bedbdec, 477944b8c22d4e01836baf0bbba0da78; no voters:
I20260806 18:38:58.352545 7072 raft_consensus.cc:2781] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:58.352658 7072 raft_consensus.cc:482] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:58.352722 7072 raft_consensus.cc:3037] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:58.353847 7072 raft_consensus.cc:504] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 35867 } } peers { permanent_uuid: "477944b8c22d4e01836baf0bbba0da78" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43991 } } peers { permanent_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44429 } }
I20260806 18:38:58.354231 7072 leader_election.cc:288] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [CANDIDATE]: Term 1 election: Requested vote from peers 477944b8c22d4e01836baf0bbba0da78 (127.18.125.131:43991), 73a7462c7b9f4581b2c4e2f2d0d2f29b (127.18.125.129:44429)
I20260806 18:38:58.354652 6999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093517769e6142a7bbec7cfb0ad7250b" candidate_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477944b8c22d4e01836baf0bbba0da78"
I20260806 18:38:58.354770 6717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093517769e6142a7bbec7cfb0ad7250b" candidate_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b"
I20260806 18:38:58.354818 6999 raft_consensus.cc:3037] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:58.354930 6717 raft_consensus.cc:3037] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:58.356014 6999 raft_consensus.cc:2454] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43b5faf26d4e4a87bcc115a19bedbdec in term 1.
I20260806 18:38:58.356374 6717 raft_consensus.cc:2454] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43b5faf26d4e4a87bcc115a19bedbdec in term 1.
I20260806 18:38:58.356426 6788 leader_election.cc:302] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43b5faf26d4e4a87bcc115a19bedbdec, 477944b8c22d4e01836baf0bbba0da78; no voters:
I20260806 18:38:58.356647 7072 raft_consensus.cc:2781] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:58.356892 7072 raft_consensus.cc:686] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 1 LEADER]: Becoming Leader. State: Replica: 43b5faf26d4e4a87bcc115a19bedbdec, State: Running, Role: LEADER
I20260806 18:38:58.357244 7072 consensus_queue.cc:237] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 35867 } } peers { permanent_uuid: "477944b8c22d4e01836baf0bbba0da78" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43991 } } peers { permanent_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44429 } }
I20260806 18:38:58.360697 6566 catalog_manager.cc:5495] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec reported cstate change: term changed from 0 to 1, leader changed from <none> to 43b5faf26d4e4a87bcc115a19bedbdec (127.18.125.130). New cstate: current_term: 1 leader_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b5faf26d4e4a87bcc115a19bedbdec" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 35867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "477944b8c22d4e01836baf0bbba0da78" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44429 } health_report { overall_health: UNKNOWN } } }
I20260806 18:38:58.403436 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.399566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52880 (local address 127.18.125.130:35867)
0806 18:38:58.399741 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:58.399746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:58.399762 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:38:58.399972 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:58.399975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:58.400037 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:58.400184 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:58.400442 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.400970 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.401856 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.402087 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.402202 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.402542 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:58.402998 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:58.403137 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:58.403202 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 18:38:58.403267 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 18:38:58.410745 6999 raft_consensus.cc:1261] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78 [term 1 FOLLOWER]: Refusing update from remote peer 43b5faf26d4e4a87bcc115a19bedbdec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:58.410732 6717 raft_consensus.cc:1261] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b [term 1 FOLLOWER]: Refusing update from remote peer 43b5faf26d4e4a87bcc115a19bedbdec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:58.411620 7072 consensus_queue.cc:1035] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [LEADER]: Connected to new peer: Peer: permanent_uuid: "73a7462c7b9f4581b2c4e2f2d0d2f29b" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 44429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:58.411906 7076 consensus_queue.cc:1035] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [LEADER]: Connected to new peer: Peer: permanent_uuid: "477944b8c22d4e01836baf0bbba0da78" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 43991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:58.422399 7081 mvcc.cc:205] Tried to move back new op lower bound from 7315626141320970240 to 7315626141114138624. Current Snapshot: MvccSnapshot[applied={T|T < 7315626141320970240}]
I20260806 18:38:58.436491 6568 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:58.436715 6568 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:58.439729 6568 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95b17761efed42b3ae955667095ddb6f: Sending DeleteTablet for 3 replicas of tablet 093517769e6142a7bbec7cfb0ad7250b
I20260806 18:38:58.440729 6835 tablet_service.cc:1514] Processing DeleteTablet for tablet 093517769e6142a7bbec7cfb0ad7250b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:58 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:52872
I20260806 18:38:58.440817 6697 tablet_service.cc:1514] Processing DeleteTablet for tablet 093517769e6142a7bbec7cfb0ad7250b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:58 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:51462
I20260806 18:38:58.440800 6978 tablet_service.cc:1514] Processing DeleteTablet for tablet 093517769e6142a7bbec7cfb0ad7250b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:58 UTC) from {username='kudu', principal='kudu/127.18.125.190@KRBTEST.COM'} at 127.0.0.1:37246
I20260806 18:38:58.441360 7090 tablet_replica.cc:330] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b: stopping tablet replica
I20260806 18:38:58.441382 7092 tablet_replica.cc:330] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: stopping tablet replica
I20260806 18:38:58.441501 7089 tablet_replica.cc:330] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: stopping tablet replica
I20260806 18:38:58.441720 7092 raft_consensus.cc:2229] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:58.441720 7090 raft_consensus.cc:2229] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:58.441855 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 6632
I20260806 18:38:58.441855 7089 raft_consensus.cc:2229] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:38:58.442080 7092 raft_consensus.cc:2258] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:58.442081 7090 raft_consensus.cc:2258] T 093517769e6142a7bbec7cfb0ad7250b P 73a7462c7b9f4581b2c4e2f2d0d2f29b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:58.442451 7089 raft_consensus.cc:2258] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:58.444334 7092 ts_tablet_manager.cc:1905] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:58.447559 7089 ts_tablet_manager.cc:1905] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:58.447607 7092 ts_tablet_manager.cc:1918] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:58.447710 7092 log.cc:1196] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/093517769e6142a7bbec7cfb0ad7250b
I20260806 18:38:58.448076 7092 ts_tablet_manager.cc:1939] T 093517769e6142a7bbec7cfb0ad7250b P 477944b8c22d4e01836baf0bbba0da78: Deleting consensus metadata
I20260806 18:38:58.449545 6551 catalog_manager.cc:4841] TS 477944b8c22d4e01836baf0bbba0da78 (127.18.125.131:43991): tablet 093517769e6142a7bbec7cfb0ad7250b (table test-table [id=78e9a1931b644faa83f26e973e66ca37]) successfully deleted
I20260806 18:38:58.450592 7089 ts_tablet_manager.cc:1918] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:58.450819 7089 log.cc:1196] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/093517769e6142a7bbec7cfb0ad7250b
I20260806 18:38:58.451138 7089 ts_tablet_manager.cc:1939] T 093517769e6142a7bbec7cfb0ad7250b P 43b5faf26d4e4a87bcc115a19bedbdec: Deleting consensus metadata
I20260806 18:38:58.452466 6552 catalog_manager.cc:4841] TS 43b5faf26d4e4a87bcc115a19bedbdec (127.18.125.130:35867): tablet 093517769e6142a7bbec7cfb0ad7250b (table test-table [id=78e9a1931b644faa83f26e973e66ca37]) successfully deleted
W20260806 18:38:58.452809 6551 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:44429 (error 108)
I20260806 18:38:58.453114 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 6771
I20260806 18:38:58.455662 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.455252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:44429 (local address 127.0.0.1:51474)
0806 18:38:58.455435 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:58.455545 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:44429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 18:38:58.455859 6551 catalog_manager.cc:4584] TS 73a7462c7b9f4581b2c4e2f2d0d2f29b (127.18.125.129:44429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 093517769e6142a7bbec7cfb0ad7250b: Network error: Client connection negotiation failed: client connection to 127.18.125.129:44429: connect: Connection refused (error 111)
I20260806 18:38:58.461313 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 6913
I20260806 18:38:58.468649 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 6535
2026-08-06T18:38:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2703 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T18:38:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:38:58Z Disabled control of system clock
I20260806 18:38:58.502219 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:44671
--webserver_interface=127.18.125.190
--webserver_port=0
--builtin_ntp_servers=127.18.125.148:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:44671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:58.618386 7097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:58.618678 7097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:58.618716 7097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:58.622713 7097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:38:58.622764 7097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:58.622781 7097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:58.622799 7097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:38:58.622814 7097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:38:58.627521 7097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:44671
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:44671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:58.628715 7097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:58.630143 7097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:58.638880 7105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:58.638880 7102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:58.638963 7103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:58.639088 7097 server_base.cc:1034] running on GCE node
I20260806 18:38:58.639752 7097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:58.640930 7097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:58.642095 7097 hybrid_clock.cc:648] HybridClock initialized: now 1786041538642069 us; error 40 us; skew 500 ppm
I20260806 18:38:58.649371 7097 webserver.cc:460] Webserver started at http://127.18.125.190:41697/ using document root <none> and password file <none>
I20260806 18:38:58.650071 7097 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:58.650143 7097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:58.650390 7097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:58.652699 7097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "0a672b542abe47f18cfe011581d13327"
format_stamp: "Formatted at 2026-08-06 18:38:58 on dist-test-slave-8w9v"
server_key: "b16319e56fb21a0087cdbf21fcc57a71"
server_key_iv: "60c0f515d1f1a11b0c06bddc44631ce1"
server_key_version: "encryptionkey@0"
I20260806 18:38:58.653291 7097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "0a672b542abe47f18cfe011581d13327"
format_stamp: "Formatted at 2026-08-06 18:38:58 on dist-test-slave-8w9v"
server_key: "b16319e56fb21a0087cdbf21fcc57a71"
server_key_iv: "60c0f515d1f1a11b0c06bddc44631ce1"
server_key_version: "encryptionkey@0"
I20260806 18:38:58.657330 7097 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:38:58.659987 7111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:58.661334 7097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:58.661499 7097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "0a672b542abe47f18cfe011581d13327"
format_stamp: "Formatted at 2026-08-06 18:38:58 on dist-test-slave-8w9v"
server_key: "b16319e56fb21a0087cdbf21fcc57a71"
server_key_iv: "60c0f515d1f1a11b0c06bddc44631ce1"
server_key_version: "encryptionkey@0"
I20260806 18:38:58.661621 7097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:58.679001 7097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:58.679811 7097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:58.680028 7097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:58.687885 7097 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:44671
I20260806 18:38:58.687971 7165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:44671 every 8 connection(s)
I20260806 18:38:58.688943 7097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:38:58.689172 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 7097
I20260806 18:38:58.689332 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:38:58.689709 18934 external_mini_cluster.cc:1428] Setting key 9b4933cf4598302aade7950bd6ef505b
I20260806 18:38:58.692703 7166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:58.699796 7169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.691348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54942 (local address 127.18.125.190:44671)
0806 18:38:58.692044 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:58.692061 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:58.692099 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:38:58.692974 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:58.692994 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:58.693411 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:58.693732 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:58.694149 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.695130 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.696414 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.696903 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.697026 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.698002 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:58.698007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:58.698019 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:58.698423 (+ 404us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:58.698478 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:58.698498 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:58.698578 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:58.698766 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:38:58.699072 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":172,"threads_started":1}
I20260806 18:38:58.699898 7166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327: Bootstrap starting.
I20260806 18:38:58.702404 7166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:58.703205 7166 log.cc:824] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:58.705416 7166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327: No bootstrap required, opened a new log
I20260806 18:38:58.710053 7166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a672b542abe47f18cfe011581d13327" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:38:58.710321 7166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:58.710390 7166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a672b542abe47f18cfe011581d13327, State: Initialized, Role: FOLLOWER
I20260806 18:38:58.710855 7166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a672b542abe47f18cfe011581d13327" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:38:58.711010 7166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:38:58.711078 7166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:38:58.711202 7166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:58.712337 7166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a672b542abe47f18cfe011581d13327" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:38:58.712715 7166 leader_election.cc:302] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a672b542abe47f18cfe011581d13327; no voters:
I20260806 18:38:58.713094 7166 leader_election.cc:288] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:38:58.713260 7171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:58.713531 7171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [term 1 LEADER]: Becoming Leader. State: Replica: 0a672b542abe47f18cfe011581d13327, State: Running, Role: LEADER
I20260806 18:38:58.713855 7171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a672b542abe47f18cfe011581d13327" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } }
I20260806 18:38:58.714555 7166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:38:58.715739 7172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a672b542abe47f18cfe011581d13327" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a672b542abe47f18cfe011581d13327" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } } }
I20260806 18:38:58.715900 7172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:58.716248 7173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a672b542abe47f18cfe011581d13327. Latest consensus state: current_term: 1 leader_uuid: "0a672b542abe47f18cfe011581d13327" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a672b542abe47f18cfe011581d13327" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 44671 } } }
I20260806 18:38:58.716360 7173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327 [sys.catalog]: This master's current role is: LEADER
I20260806 18:38:58.716480 7180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:38:58.719903 7180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:38:58.725780 7180 catalog_manager.cc:1347] Generated new cluster ID: dc5ceb8313b7453da2855770390eb1e1
I20260806 18:38:58.725859 7180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:38:58.736393 7180 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:38:58.737793 7180 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:38:58.762176 7180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327: Generated new TSK 0
I20260806 18:38:58.763132 7180 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:38:58.769352 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.129:0
--local_ip_for_outbound_sockets=127.18.125.129
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--builtin_ntp_servers=127.18.125.148:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:38:58.901595 7191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:58.901897 7191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:58.901940 7191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:58.906205 7191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:58.906280 7191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:58.906339 7191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.129
I20260806 18:38:58.911471 7191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.125.129
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.129
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:58.912798 7191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:58.914299 7191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:58.925048 7199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:58.925163 7197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:58.925009 7196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:58.925065 7191 server_base.cc:1034] running on GCE node
I20260806 18:38:58.925839 7191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:58.926570 7191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:58.927790 7191 hybrid_clock.cc:648] HybridClock initialized: now 1786041538927759 us; error 34 us; skew 500 ppm
I20260806 18:38:58.930083 7191 webserver.cc:460] Webserver started at http://127.18.125.129:42127/ using document root <none> and password file <none>
I20260806 18:38:58.930754 7191 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:58.930861 7191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:58.931142 7191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:58.933977 7191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/data/instance:
uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74"
format_stamp: "Formatted at 2026-08-06 18:38:58 on dist-test-slave-8w9v"
server_key: "0bf80a41822712523288136aec45bfed"
server_key_iv: "a7a74b4e2fed2facc8901d102b430b9d"
server_key_version: "encryptionkey@0"
I20260806 18:38:58.934640 7191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance:
uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74"
format_stamp: "Formatted at 2026-08-06 18:38:58 on dist-test-slave-8w9v"
server_key: "0bf80a41822712523288136aec45bfed"
server_key_iv: "a7a74b4e2fed2facc8901d102b430b9d"
server_key_version: "encryptionkey@0"
I20260806 18:38:58.939541 7191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 18:38:58.942538 7205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:58.943953 7191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 18:38:58.944171 7191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/wal
uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74"
format_stamp: "Formatted at 2026-08-06 18:38:58 on dist-test-slave-8w9v"
server_key: "0bf80a41822712523288136aec45bfed"
server_key_iv: "a7a74b4e2fed2facc8901d102b430b9d"
server_key_version: "encryptionkey@0"
I20260806 18:38:58.944340 7191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:58.960719 7191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:58.961532 7191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:58.961756 7191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:58.962427 7191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:58.963464 7191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:58.963538 7191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:58.963611 7191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:58.963662 7191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:58.973574 7191 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.129:33389
I20260806 18:38:58.973603 7325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.129:33389 every 8 connection(s)
I20260806 18:38:58.974581 7191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/data/info.pb
I20260806 18:38:58.977787 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 7191
I20260806 18:38:58.977931 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-0/wal/instance
I20260806 18:38:58.978299 18934 external_mini_cluster.cc:1428] Setting key 21d2206ba80d387818a23940c66f95c7
I20260806 18:38:58.981091 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.130:0
--local_ip_for_outbound_sockets=127.18.125.130
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--builtin_ntp_servers=127.18.125.148:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:58.986531 7169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.976714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.129:37169 (local address 127.18.125.190:44671)
0806 18:38:58.976874 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:58.976878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:58.978295 (+ 1417us) server_negotiation.cc:407] Connection header received
0806 18:38:58.980144 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:58.980149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:58.980233 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:58.980344 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:58.981706 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.982285 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.983762 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.984221 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.984359 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.985784 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:58.985791 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:58.985798 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:58.985820 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:58.985854 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:58.985868 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:58.985976 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:58.986175 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 18:38:58.986299 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:38:58.987386 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:58.977167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.129:37169)
0806 18:38:58.978086 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:58.978153 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:58.979821 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:58.980592 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:58.980603 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:58.980725 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:58.981518 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.981546 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.982454 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.982460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.983512 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:58.983524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:58.984389 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:58.984392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:58.984565 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:58.985390 (+ 825us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:58.985392 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:58.985439 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:58.985520 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:58.985996 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:58.986011 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:58.986017 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:38:58.986129 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:38:58.986421 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":147,"threads_started":1}
I20260806 18:38:58.988981 7326 heartbeater.cc:347] Connected to a master server at 127.18.125.190:44671
I20260806 18:38:58.989270 7326 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:58.989859 7326 heartbeater.cc:511] Master 127.18.125.190:44671 requested a full tablet report, sending...
I20260806 18:38:58.991474 7128 ts_manager.cc:194] Registered new tserver with Master: b9d9fa8e0b2e43828053d2ee7bcdeb74 (127.18.125.129:33389)
I20260806 18:38:58.992791 7128 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.129:37169
W20260806 18:38:59.103798 7329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:59.104105 7329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:59.104255 7329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:59.108536 7329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:59.108605 7329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:59.108664 7329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.130
I20260806 18:38:59.113701 7329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.125.130
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.130
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:59.115042 7329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:59.116676 7329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:59.127153 7336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:59.127190 7337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:59.127194 7340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:59.127571 7329 server_base.cc:1034] running on GCE node
I20260806 18:38:59.128005 7329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:59.128747 7329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:59.129923 7329 hybrid_clock.cc:648] HybridClock initialized: now 1786041539129887 us; error 52 us; skew 500 ppm
I20260806 18:38:59.131935 7329 webserver.cc:460] Webserver started at http://127.18.125.130:45275/ using document root <none> and password file <none>
I20260806 18:38:59.132604 7329 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:59.132669 7329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:59.132893 7329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:59.135044 7329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/data/instance:
uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e"
format_stamp: "Formatted at 2026-08-06 18:38:59 on dist-test-slave-8w9v"
server_key: "310053a8e158bdbb73509e547acbee9c"
server_key_iv: "e27a7404829af28227ad3b1ef3a5fbd6"
server_key_version: "encryptionkey@0"
I20260806 18:38:59.135643 7329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance:
uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e"
format_stamp: "Formatted at 2026-08-06 18:38:59 on dist-test-slave-8w9v"
server_key: "310053a8e158bdbb73509e547acbee9c"
server_key_iv: "e27a7404829af28227ad3b1ef3a5fbd6"
server_key_version: "encryptionkey@0"
I20260806 18:38:59.139760 7329 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 18:38:59.142485 7347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:59.143908 7329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 18:38:59.144068 7329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/wal
uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e"
format_stamp: "Formatted at 2026-08-06 18:38:59 on dist-test-slave-8w9v"
server_key: "310053a8e158bdbb73509e547acbee9c"
server_key_iv: "e27a7404829af28227ad3b1ef3a5fbd6"
server_key_version: "encryptionkey@0"
I20260806 18:38:59.144250 7329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:59.171252 7329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:59.172009 7329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:59.172263 7329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:59.172914 7329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:59.173925 7329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:59.173996 7329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:59.174069 7329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:59.174113 7329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:59.183321 7329 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.130:38983
I20260806 18:38:59.183353 7462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.130:38983 every 8 connection(s)
I20260806 18:38:59.184312 7329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/data/info.pb
I20260806 18:38:59.189579 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 7329
I20260806 18:38:59.189760 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/wal/instance
I20260806 18:38:59.190070 18934 external_mini_cluster.cc:1428] Setting key 1b2a7982cb729791597ab47e50e1c4b6
I20260806 18:38:59.192091 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.125.131:0
--local_ip_for_outbound_sockets=127.18.125.131
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--builtin_ntp_servers=127.18.125.148:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:38:59.194862 7169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.186265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:39929 (local address 127.18.125.190:44671)
0806 18:38:59.186419 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:59.186423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:59.187467 (+ 1044us) server_negotiation.cc:407] Connection header received
0806 18:38:59.188942 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:59.188946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:59.189011 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:59.189092 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:59.190099 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.190777 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.192054 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.192422 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.192525 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.194157 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:59.194164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:59.194168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:59.194189 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:59.194216 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:59.194229 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:59.194279 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:59.194545 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 18:38:59.194640 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:38:59.195649 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.186649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.130:39929)
0806 18:38:59.187294 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:59.187334 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:59.188715 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:59.189278 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:59.189285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:59.189369 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:59.189955 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.189970 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.190901 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.190904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.191928 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.191938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.192600 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.192607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.192860 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.193775 (+ 915us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:59.193780 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:59.193842 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:59.193942 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:59.194352 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:59.194360 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:59.194364 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:38:59.194475 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 18:38:59.194729 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":122,"threads_started":1}
I20260806 18:38:59.197101 7463 heartbeater.cc:347] Connected to a master server at 127.18.125.190:44671
I20260806 18:38:59.197404 7463 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:59.197994 7463 heartbeater.cc:511] Master 127.18.125.190:44671 requested a full tablet report, sending...
I20260806 18:38:59.199369 7128 ts_manager.cc:194] Registered new tserver with Master: d7871d4642ac4ce2b9c416bce5fe6d0e (127.18.125.130:38983)
I20260806 18:38:59.200003 7128 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.130:39929
W20260806 18:38:59.327255 7466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:38:59.327545 7466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:38:59.327589 7466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:38:59.332305 7466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:38:59.332391 7466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:38:59.332495 7466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.125.131
I20260806 18:38:59.337744 7466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.125.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.125.131
--webserver_port=0
--tserver_master_addrs=127.18.125.190:44671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.125.131
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:38:59.339018 7466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:38:59.340561 7466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:38:59.351202 7474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:59.351233 7473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:38:59.351203 7476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:38:59.351609 7466 server_base.cc:1034] running on GCE node
I20260806 18:38:59.352074 7466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:38:59.352907 7466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:38:59.354130 7466 hybrid_clock.cc:648] HybridClock initialized: now 1786041539354105 us; error 43 us; skew 500 ppm
I20260806 18:38:59.356478 7466 webserver.cc:460] Webserver started at http://127.18.125.131:39743/ using document root <none> and password file <none>
I20260806 18:38:59.357159 7466 fs_manager.cc:359] Metadata directory not provided
I20260806 18:38:59.357229 7466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:38:59.357465 7466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:38:59.359709 7466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/data/instance:
uuid: "1389e495c97f4a4fa431e717dc8673fb"
format_stamp: "Formatted at 2026-08-06 18:38:59 on dist-test-slave-8w9v"
server_key: "f8a48ed0080240e33e95d6819b703dfb"
server_key_iv: "a5fcb8fc9fbbe4f11c23d907d619c802"
server_key_version: "encryptionkey@0"
I20260806 18:38:59.360354 7466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance:
uuid: "1389e495c97f4a4fa431e717dc8673fb"
format_stamp: "Formatted at 2026-08-06 18:38:59 on dist-test-slave-8w9v"
server_key: "f8a48ed0080240e33e95d6819b703dfb"
server_key_iv: "a5fcb8fc9fbbe4f11c23d907d619c802"
server_key_version: "encryptionkey@0"
I20260806 18:38:59.364545 7466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:38:59.367535 7482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:59.368989 7466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:38:59.369127 7466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/wal
uuid: "1389e495c97f4a4fa431e717dc8673fb"
format_stamp: "Formatted at 2026-08-06 18:38:59 on dist-test-slave-8w9v"
server_key: "f8a48ed0080240e33e95d6819b703dfb"
server_key_iv: "a5fcb8fc9fbbe4f11c23d907d619c802"
server_key_version: "encryptionkey@0"
I20260806 18:38:59.369251 7466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:38:59.390416 7466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:38:59.391281 7466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:38:59.391518 7466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:38:59.392289 7466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:38:59.393520 7466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:38:59.393615 7466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:59.393716 7466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:38:59.393754 7466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:38:59.405606 7466 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.131:35601
I20260806 18:38:59.405689 7602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.131:35601 every 8 connection(s)
I20260806 18:38:59.406672 7466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/data/info.pb
I20260806 18:38:59.412150 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 7466
I20260806 18:38:59.412298 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/wal/instance
I20260806 18:38:59.412693 18934 external_mini_cluster.cc:1428] Setting key d28ea4fa22286ac914bffcabb15a17d1
I20260806 18:38:59.421326 7169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.408981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.131:50727 (local address 127.18.125.190:44671)
0806 18:38:59.409139 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:59.409143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:59.410789 (+ 1646us) server_negotiation.cc:407] Connection header received
0806 18:38:59.412515 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:59.412520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:59.412596 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:59.412725 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:59.414082 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.414928 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.417784 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.418030 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.418140 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.420419 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:59.420423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:59.420425 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:59.420443 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:59.420473 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:59.420484 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:59.420562 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:59.420883 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 18:38:59.421132 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:38:59.422916 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.409647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:44671 (local address 127.18.125.131:50727)
0806 18:38:59.410580 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:59.410629 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:59.412098 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:59.412956 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:59.412965 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:59.413061 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:38:59.413914 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.413941 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.415130 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.415135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.417519 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.417545 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.418401 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.418409 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.418722 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.419835 (+ 1113us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:38:59.419839 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:38:59.419936 (+ 97us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:38:59.420099 (+ 163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:38:59.420617 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:38:59.420631 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:38:59.420642 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 18:38:59.420824 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 18:38:59.421521 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":197,"threads_started":1}
I20260806 18:38:59.424788 7603 heartbeater.cc:347] Connected to a master server at 127.18.125.190:44671
I20260806 18:38:59.425118 7603 heartbeater.cc:464] Registering TS with master...
I20260806 18:38:59.425797 7603 heartbeater.cc:511] Master 127.18.125.190:44671 requested a full tablet report, sending...
I20260806 18:38:59.427153 7128 ts_manager.cc:194] Registered new tserver with Master: 1389e495c97f4a4fa431e717dc8673fb (127.18.125.131:35601)
I20260806 18:38:59.427983 18934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:38:59.428014 7128 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.125.131:50727
I20260806 18:38:59.434928 7169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.430830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.18.125.190:44671)
0806 18:38:59.430993 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:59.431000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:59.431207 (+ 207us) server_negotiation.cc:407] Connection header received
0806 18:38:59.431371 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:59.431376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:59.431432 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:59.431524 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:38:59.431826 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.432410 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.433291 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.433512 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.433623 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.434476 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:38:59.434481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:38:59.434485 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:38:59.434503 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:38:59.434520 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:38:59.434527 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:38:59.434607 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:59.434698 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 18:38:59.434770 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:38:59.439788 7128 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:38:59.445051 7128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:38:59.466084 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.457612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:33389 (local address 127.0.0.1:46574)
0806 18:38:59.459636 (+ 2024us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:59.459662 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:59.459781 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:59.460605 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:59.460611 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:59.460636 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:59.460824 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.460831 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.463165 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.463168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.464351 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.464364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.465735 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.465739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.465830 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.465850 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:38:59.465920 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:38:59.465980 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1932,"thread_start_us":118,"threads_started":1}
I20260806 18:38:59.466747 7619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.458632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54446 (local address 127.18.125.131:35601)
0806 18:38:59.459006 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:59.459021 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:59.459555 (+ 534us) server_negotiation.cc:407] Connection header received
0806 18:38:59.459957 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:59.459964 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:59.460470 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:59.460633 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:59.461062 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.462164 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.463542 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.464937 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.465085 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.465281 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:59.466167 (+ 886us) server_negotiation.cc:299] Negotiation successful
0806 18:38:59.466299 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":148,"threads_started":1}
I20260806 18:38:59.466053 7620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.457584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:35601 (local address 127.0.0.1:54446)
0806 18:38:59.459350 (+ 1766us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:59.459443 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:59.459716 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:59.460824 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:59.460827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:59.460843 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:59.460938 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.460944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.462296 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.462301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.463388 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.463399 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.465415 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.465421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.465551 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.465593 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 18:38:59.465715 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 18:38:59.465853 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1592,"thread_start_us":130,"threads_started":1}
I20260806 18:38:59.467653 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.457878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.18.125.129:33389)
0806 18:38:59.459457 (+ 1579us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:59.459466 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:59.459922 (+ 456us) server_negotiation.cc:407] Connection header received
0806 18:38:59.460058 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:59.460066 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:59.460386 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:59.461428 (+ 1042us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:59.461641 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.463012 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.464523 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.465381 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.465504 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.465669 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:59.466568 (+ 899us) server_negotiation.cc:299] Negotiation successful
0806 18:38:59.466684 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":133,"threads_started":1}
I20260806 18:38:59.470395 7536 tablet_service.cc:1467] Processing CreateTablet for tablet 767f7dc0ba89479894f7d8272ac3adbf (DEFAULT_TABLE table=test-table [id=92403ad7a27045bdb4434d801b8d3534]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:59.470996 7257 tablet_service.cc:1467] Processing CreateTablet for tablet 767f7dc0ba89479894f7d8272ac3adbf (DEFAULT_TABLE table=test-table [id=92403ad7a27045bdb4434d801b8d3534]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:59.471508 7536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 767f7dc0ba89479894f7d8272ac3adbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:59.472198 7257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 767f7dc0ba89479894f7d8272ac3adbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:59.475426 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.458941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:38983 (local address 127.0.0.1:46552)
0806 18:38:59.463361 (+ 4420us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:59.463381 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:59.463506 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:59.464469 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:59.464474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:59.464500 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:59.465349 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.465365 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.467540 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.467544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.473228 (+ 5684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.473250 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.474929 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.474934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.475028 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.475061 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 18:38:59.475143 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:38:59.475252 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4308,"thread_start_us":132,"threads_started":1}
I20260806 18:38:59.476491 7621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.459527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.18.125.130:38983)
0806 18:38:59.460338 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:59.460355 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:59.463632 (+ 3277us) server_negotiation.cc:407] Connection header received
0806 18:38:59.463752 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:59.463759 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:59.464057 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:59.464254 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:59.465553 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.467397 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.473490 (+ 6093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.474490 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.474639 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.474840 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:59.476186 (+ 1346us) server_negotiation.cc:299] Negotiation successful
0806 18:38:59.476316 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":175,"threads_started":1}
I20260806 18:38:59.479707 7622 tablet_bootstrap.cc:492] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: Bootstrap starting.
I20260806 18:38:59.480223 7395 tablet_service.cc:1467] Processing CreateTablet for tablet 767f7dc0ba89479894f7d8272ac3adbf (DEFAULT_TABLE table=test-table [id=92403ad7a27045bdb4434d801b8d3534]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:38:59.481674 7395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 767f7dc0ba89479894f7d8272ac3adbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:38:59.482601 7622 tablet_bootstrap.cc:654] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:59.483604 7622 log.cc:824] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:59.489847 7624 tablet_bootstrap.cc:492] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74: Bootstrap starting.
I20260806 18:38:59.490211 7622 tablet_bootstrap.cc:492] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: No bootstrap required, opened a new log
I20260806 18:38:59.490535 7622 ts_tablet_manager.cc:1397] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260806 18:38:59.492542 7625 tablet_bootstrap.cc:492] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: Bootstrap starting.
I20260806 18:38:59.493309 7624 tablet_bootstrap.cc:654] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:59.494474 7624 log.cc:824] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:59.495826 7625 tablet_bootstrap.cc:654] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: Neither blocks nor log segments found. Creating new log.
I20260806 18:38:59.497202 7625 log.cc:824] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:38:59.497606 7622 raft_consensus.cc:348] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1389e495c97f4a4fa431e717dc8673fb" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 35601 } } peers { permanent_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 33389 } } peers { permanent_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38983 } }
I20260806 18:38:59.497951 7624 tablet_bootstrap.cc:492] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74: No bootstrap required, opened a new log
I20260806 18:38:59.498142 7622 raft_consensus.cc:374] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:59.498201 7624 ts_tablet_manager.cc:1397] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260806 18:38:59.498322 7622 raft_consensus.cc:729] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1389e495c97f4a4fa431e717dc8673fb, State: Initialized, Role: FOLLOWER
I20260806 18:38:59.499060 7622 consensus_queue.cc:260] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1389e495c97f4a4fa431e717dc8673fb" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 35601 } } peers { permanent_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 33389 } } peers { permanent_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38983 } }
I20260806 18:38:59.501168 7622 ts_tablet_manager.cc:1428] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260806 18:38:59.501951 7603 heartbeater.cc:503] Master 127.18.125.190:44671 was elected leader, sending a full tablet report...
I20260806 18:38:59.501967 7625 tablet_bootstrap.cc:492] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: No bootstrap required, opened a new log
I20260806 18:38:59.502225 7625 ts_tablet_manager.cc:1397] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260806 18:38:59.505688 7624 raft_consensus.cc:348] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1389e495c97f4a4fa431e717dc8673fb" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 35601 } } peers { permanent_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 33389 } } peers { permanent_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38983 } }
I20260806 18:38:59.506168 7624 raft_consensus.cc:374] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:59.506275 7624 raft_consensus.cc:729] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9d9fa8e0b2e43828053d2ee7bcdeb74, State: Initialized, Role: FOLLOWER
I20260806 18:38:59.506894 7624 consensus_queue.cc:260] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1389e495c97f4a4fa431e717dc8673fb" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 35601 } } peers { permanent_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 33389 } } peers { permanent_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38983 } }
I20260806 18:38:59.507929 7625 raft_consensus.cc:348] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1389e495c97f4a4fa431e717dc8673fb" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 35601 } } peers { permanent_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 33389 } } peers { permanent_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38983 } }
I20260806 18:38:59.508361 7625 raft_consensus.cc:374] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:38:59.508443 7625 raft_consensus.cc:729] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7871d4642ac4ce2b9c416bce5fe6d0e, State: Initialized, Role: FOLLOWER
I20260806 18:38:59.508563 7624 ts_tablet_manager.cc:1428] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260806 18:38:59.509047 7326 heartbeater.cc:503] Master 127.18.125.190:44671 was elected leader, sending a full tablet report...
I20260806 18:38:59.509045 7625 consensus_queue.cc:260] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1389e495c97f4a4fa431e717dc8673fb" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 35601 } } peers { permanent_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 33389 } } peers { permanent_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38983 } }
I20260806 18:38:59.509832 7463 heartbeater.cc:503] Master 127.18.125.190:44671 was elected leader, sending a full tablet report...
I20260806 18:38:59.510206 7625 ts_tablet_manager.cc:1428] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260806 18:38:59.657960 7604 tablet.cc:2369] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:59.660795 7630 raft_consensus.cc:482] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:38:59.661065 7630 raft_consensus.cc:504] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1389e495c97f4a4fa431e717dc8673fb" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 35601 } } peers { permanent_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 33389 } } peers { permanent_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38983 } }
I20260806 18:38:59.662253 7630 leader_election.cc:288] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1389e495c97f4a4fa431e717dc8673fb (127.18.125.131:35601), b9d9fa8e0b2e43828053d2ee7bcdeb74 (127.18.125.129:33389)
I20260806 18:38:59.666980 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.662568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.131:35601 (local address 127.18.125.130:43729)
0806 18:38:59.662725 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:59.662743 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:59.662869 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:59.663211 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:59.663215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:59.663235 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:59.663362 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.663371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.664393 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.664397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.665601 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.665613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.666544 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.666547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.666618 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.666640 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:38:59.666709 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:38:59.666773 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 18:38:59.667016 7619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.662689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:43729 (local address 127.18.125.131:35601)
0806 18:38:59.662846 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:59.662852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:59.662874 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:38:59.663014 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:59.663020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:59.663085 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:59.663184 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:59.663483 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.664275 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.665771 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.666403 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.666540 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.666601 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:59.666758 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 18:38:59.666855 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:38:59.667415 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.662805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.125.130:60835 (local address 127.18.125.129:33389)
0806 18:38:59.662977 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:59.662982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:59.663340 (+ 358us) server_negotiation.cc:407] Connection header received
0806 18:38:59.663506 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:59.663510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:59.663570 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:59.663665 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:38:59.663972 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.664633 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.666110 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.666741 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.666870 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.666919 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:59.667174 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 18:38:59.667262 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:38:59.667806 7631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.662573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:33389 (local address 127.18.125.130:60835)
0806 18:38:59.663242 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:59.663255 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:38:59.663345 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:38:59.663714 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:38:59.663716 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:38:59.663735 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:38:59.663842 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.663847 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.664774 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.664778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.665966 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:38:59.665975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.666889 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.666892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:38:59.666996 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.667031 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 18:38:59.667579 (+ 548us) client_negotiation.cc:240] Negotiation successful
0806 18:38:59.667668 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":297,"thread_start_us":105,"threads_started":1}
I20260806 18:38:59.667833 7556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "767f7dc0ba89479894f7d8272ac3adbf" candidate_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1389e495c97f4a4fa431e717dc8673fb" is_pre_election: true
I20260806 18:38:59.668236 7556 raft_consensus.cc:2454] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7871d4642ac4ce2b9c416bce5fe6d0e in term 0.
I20260806 18:38:59.668527 7277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "767f7dc0ba89479894f7d8272ac3adbf" candidate_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" is_pre_election: true
I20260806 18:38:59.668813 7277 raft_consensus.cc:2454] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7871d4642ac4ce2b9c416bce5fe6d0e in term 0.
I20260806 18:38:59.668869 7348 leader_election.cc:302] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1389e495c97f4a4fa431e717dc8673fb, d7871d4642ac4ce2b9c416bce5fe6d0e; no voters:
I20260806 18:38:59.669157 7630 raft_consensus.cc:2781] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:38:59.669250 7630 raft_consensus.cc:482] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:38:59.669314 7630 raft_consensus.cc:3037] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:59.670418 7630 raft_consensus.cc:504] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1389e495c97f4a4fa431e717dc8673fb" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 35601 } } peers { permanent_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 33389 } } peers { permanent_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38983 } }
I20260806 18:38:59.670826 7630 leader_election.cc:288] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [CANDIDATE]: Term 1 election: Requested vote from peers 1389e495c97f4a4fa431e717dc8673fb (127.18.125.131:35601), b9d9fa8e0b2e43828053d2ee7bcdeb74 (127.18.125.129:33389)
I20260806 18:38:59.671209 7556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "767f7dc0ba89479894f7d8272ac3adbf" candidate_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1389e495c97f4a4fa431e717dc8673fb"
I20260806 18:38:59.671279 7277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "767f7dc0ba89479894f7d8272ac3adbf" candidate_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74"
I20260806 18:38:59.671339 7556 raft_consensus.cc:3037] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:59.671454 7277 raft_consensus.cc:3037] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:38:59.672662 7556 raft_consensus.cc:2454] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7871d4642ac4ce2b9c416bce5fe6d0e in term 1.
I20260806 18:38:59.672848 7277 raft_consensus.cc:2454] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7871d4642ac4ce2b9c416bce5fe6d0e in term 1.
I20260806 18:38:59.673106 7348 leader_election.cc:302] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1389e495c97f4a4fa431e717dc8673fb, d7871d4642ac4ce2b9c416bce5fe6d0e; no voters:
I20260806 18:38:59.673307 7630 raft_consensus.cc:2781] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:38:59.673553 7630 raft_consensus.cc:686] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 1 LEADER]: Becoming Leader. State: Replica: d7871d4642ac4ce2b9c416bce5fe6d0e, State: Running, Role: LEADER
I20260806 18:38:59.673946 7630 consensus_queue.cc:237] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1389e495c97f4a4fa431e717dc8673fb" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 35601 } } peers { permanent_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 33389 } } peers { permanent_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38983 } }
I20260806 18:38:59.677518 7128 catalog_manager.cc:5495] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e reported cstate change: term changed from 0 to 1, leader changed from <none> to d7871d4642ac4ce2b9c416bce5fe6d0e (127.18.125.130). New cstate: current_term: 1 leader_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1389e495c97f4a4fa431e717dc8673fb" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 35601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 33389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7871d4642ac4ce2b9c416bce5fe6d0e" member_type: VOTER last_known_addr { host: "127.18.125.130" port: 38983 } health_report { overall_health: HEALTHY } } }
W20260806 18:38:59.685514 7464 tablet.cc:2369] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:59.714675 7621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.710897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46562 (local address 127.18.125.130:38983)
0806 18:38:59.711063 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 18:38:59.711071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:38:59.711088 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:38:59.711180 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:38:59.711183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:38:59.711240 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:38:59.711356 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:38:59.711620 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.712159 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.713033 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:38:59.713250 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:38:59.713364 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:38:59.713689 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:38:59.714221 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:38:59.714357 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 18:38:59.714437 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 18:38:59.714495 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:38:59.720835 7556 raft_consensus.cc:1261] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb [term 1 FOLLOWER]: Refusing update from remote peer d7871d4642ac4ce2b9c416bce5fe6d0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:59.720835 7277 raft_consensus.cc:1261] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74 [term 1 FOLLOWER]: Refusing update from remote peer d7871d4642ac4ce2b9c416bce5fe6d0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:38:59.721625 7632 consensus_queue.cc:1035] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9d9fa8e0b2e43828053d2ee7bcdeb74" member_type: VOTER last_known_addr { host: "127.18.125.129" port: 33389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:38:59.721863 7630 consensus_queue.cc:1035] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1389e495c97f4a4fa431e717dc8673fb" member_type: VOTER last_known_addr { host: "127.18.125.131" port: 35601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:38:59.726066 7327 tablet.cc:2369] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:38:59.729480 7635 mvcc.cc:205] Tried to move back new op lower bound from 7315626146689105920 to 7315626146507546624. Current Snapshot: MvccSnapshot[applied={T|T < 7315626146689105920}]
I20260806 18:38:59.729466 7637 mvcc.cc:205] Tried to move back new op lower bound from 7315626146689105920 to 7315626146507546624. Current Snapshot: MvccSnapshot[applied={T|T < 7315626146689105920}]
I20260806 18:38:59.740015 7127 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:59.740293 7127 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:38:59.742887 7127 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a672b542abe47f18cfe011581d13327: Sending DeleteTablet for 3 replicas of tablet 767f7dc0ba89479894f7d8272ac3adbf
I20260806 18:38:59.743839 7257 tablet_service.cc:1514] Processing DeleteTablet for tablet 767f7dc0ba89479894f7d8272ac3adbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:59 UTC) from {username='slave'} at 127.0.0.1:46574
I20260806 18:38:59.743837 7536 tablet_service.cc:1514] Processing DeleteTablet for tablet 767f7dc0ba89479894f7d8272ac3adbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:59 UTC) from {username='slave'} at 127.0.0.1:54446
I20260806 18:38:59.743844 7395 tablet_service.cc:1514] Processing DeleteTablet for tablet 767f7dc0ba89479894f7d8272ac3adbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:38:59 UTC) from {username='slave'} at 127.0.0.1:46552
I20260806 18:38:59.745134 7646 tablet_replica.cc:330] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: stopping tablet replica
I20260806 18:38:59.745086 7647 tablet_replica.cc:330] T 767f7dc0ba89479894f7d8272ac3adbf P b9d9fa8e0b2e43828053d2ee7bcdeb74: stopping tablet replica
I20260806 18:38:59.745466 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 7191
I20260806 18:38:59.746466 7646 raft_consensus.cc:2229] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:38:59.747030 7646 raft_consensus.cc:2258] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:59.751755 7646 ts_tablet_manager.cc:1905] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:59.752183 7648 tablet_replica.cc:330] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: stopping tablet replica
I20260806 18:38:59.752519 7648 raft_consensus.cc:2229] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:38:59.752945 7648 raft_consensus.cc:2258] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:38:59.755474 7646 ts_tablet_manager.cc:1918] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:59.755604 7646 log.cc:1196] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-2/wal/wals/767f7dc0ba89479894f7d8272ac3adbf
I20260806 18:38:59.756384 7646 ts_tablet_manager.cc:1939] T 767f7dc0ba89479894f7d8272ac3adbf P 1389e495c97f4a4fa431e717dc8673fb: Deleting consensus metadata
I20260806 18:38:59.756901 7648 ts_tablet_manager.cc:1905] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:38:59.757903 7112 catalog_manager.cc:4841] TS 1389e495c97f4a4fa431e717dc8673fb (127.18.125.131:35601): tablet 767f7dc0ba89479894f7d8272ac3adbf (table test-table [id=92403ad7a27045bdb4434d801b8d3534]) successfully deleted
W20260806 18:38:59.758893 7113 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.125.129:33389 (error 108)
I20260806 18:38:59.759724 7648 ts_tablet_manager.cc:1918] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:38:59.759831 7648 log.cc:1196] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: Deleting WAL directory at /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041269054381-18934-0/minicluster-data/ts-1/wal/wals/767f7dc0ba89479894f7d8272ac3adbf
I20260806 18:38:59.760195 7648 ts_tablet_manager.cc:1939] T 767f7dc0ba89479894f7d8272ac3adbf P d7871d4642ac4ce2b9c416bce5fe6d0e: Deleting consensus metadata
I20260806 18:38:59.760329 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.759978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.129:33389 (local address 127.0.0.1:46576)
0806 18:38:59.760097 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:59.760192 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.129:33389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 18:38:59.760468 7113 catalog_manager.cc:4584] TS b9d9fa8e0b2e43828053d2ee7bcdeb74 (127.18.125.129:33389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 767f7dc0ba89479894f7d8272ac3adbf: Network error: Client connection negotiation failed: client connection to 127.18.125.129:33389: connect: Connection refused (error 111)
I20260806 18:38:59.760895 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 7329
I20260806 18:38:59.768855 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 7466
I20260806 18:38:59.770399 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:38:59.769116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.130:38983 (local address 127.0.0.1:46564)
0806 18:38:59.770191 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0806 18:38:59.770278 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.130:38983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":960}
I20260806 18:38:59.776921 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 7097
2026-08-06T18:38:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1306 ms)
[----------] 6 tests from EncryptionPolicyTest (11365 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:38:59 dist-test-slave-8w9v krb5kdc[7652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:38:59 dist-test-slave-8w9v krb5kdc[7652](info): set up 2 sockets
Aug 06 18:38:59 dist-test-slave-8w9v krb5kdc[7652](info): commencing operation
krb5kdc: starting...
W20260806 18:39:00.871963 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:00 dist-test-slave-8w9v krb5kdc[7652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:01.008279 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43949
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:01.124274 7667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:01.124626 7667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:01.124714 7667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:01.128846 7667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:01.128961 7667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:01.128999 7667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:01.129040 7667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:01.129081 7667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:01.134598 7667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43949
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:01.135841 7667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:01.137027 7667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:01.146162 7675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:01.146143 7673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:01.146147 7672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:01.146538 7667 server_base.cc:1034] running on GCE node
I20260806 18:39:01.147149 7667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:01.148488 7667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:01.149698 7667 hybrid_clock.cc:648] HybridClock initialized: now 1786041541149675 us; error 37 us; skew 500 ppm
Aug 06 18:39:01 dist-test-slave-8w9v krb5kdc[7652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041541, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:01.152886 7667 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:01.153906 7667 webserver.cc:460] Webserver started at http://127.0.0.1:37205/ using document root <none> and password file <none>
I20260806 18:39:01.154564 7667 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:01.154671 7667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:01.155030 7667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:01.157653 7667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "709396d630a64cca90171b29673cffd2"
format_stamp: "Formatted at 2026-08-06 18:39:01 on dist-test-slave-8w9v"
server_key: "2f98389953f3d11d4a76dca2251294e0"
server_key_iv: "f277b396cf44899486704af6178dc860"
server_key_version: "encryptionkey@0"
I20260806 18:39:01.158288 7667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "709396d630a64cca90171b29673cffd2"
format_stamp: "Formatted at 2026-08-06 18:39:01 on dist-test-slave-8w9v"
server_key: "2f98389953f3d11d4a76dca2251294e0"
server_key_iv: "f277b396cf44899486704af6178dc860"
server_key_version: "encryptionkey@0"
I20260806 18:39:01.162325 7667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:39:01.165184 7683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:01.166646 7667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:01.166805 7667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "709396d630a64cca90171b29673cffd2"
format_stamp: "Formatted at 2026-08-06 18:39:01 on dist-test-slave-8w9v"
server_key: "2f98389953f3d11d4a76dca2251294e0"
server_key_iv: "f277b396cf44899486704af6178dc860"
server_key_version: "encryptionkey@0"
I20260806 18:39:01.166945 7667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:01.179999 7667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:01.183704 7667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:01.183938 7667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:01.191496 7667 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43949
I20260806 18:39:01.191521 7736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43949 every 8 connection(s)
I20260806 18:39:01.192579 7667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:01.195780 7737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:01.196411 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 7667
I20260806 18:39:01.196555 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:01.196906 18934 external_mini_cluster.cc:1428] Setting key 05b212b379d9fb37605cf6880f38beca
I20260806 18:39:01.203269 7737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2: Bootstrap starting.
I20260806 18:39:01.205737 7737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:01.206655 7737 log.cc:824] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:39:01 dist-test-slave-8w9v krb5kdc[7652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:39:01.209427 7737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2: No bootstrap required, opened a new log
I20260806 18:39:01.213277 7740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:01.198526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53885 (local address 127.0.0.1:43949)
0806 18:39:01.199030 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:01.199047 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:01.199138 (+ 91us) server_negotiation.cc:407] Connection header received
0806 18:39:01.199935 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:01.199956 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:01.200404 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:01.200790 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:01.201730 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:01.202759 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:01.203758 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:01.204338 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:01.204539 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:01.207777 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:01.207784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:01.207802 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:01.207831 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:01.209924 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:01.210447 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:01.210453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:01.210459 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:01.210602 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:01.210943 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:01.210946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:01.210948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:01.211481 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:01.211736 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:01.211999 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 18:39:01.212276 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":147,"threads_started":1}
I20260806 18:39:01.214260 7737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709396d630a64cca90171b29673cffd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43949 } }
I20260806 18:39:01.214560 7737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:01.214654 7737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 709396d630a64cca90171b29673cffd2, State: Initialized, Role: FOLLOWER
I20260806 18:39:01.215134 7737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709396d630a64cca90171b29673cffd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43949 } }
I20260806 18:39:01.215306 7737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:01.215390 7737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:01.215487 7737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:01.216727 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:01.216734 7737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709396d630a64cca90171b29673cffd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43949 } }
I20260806 18:39:01.217092 7737 leader_election.cc:302] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 709396d630a64cca90171b29673cffd2; no voters:
I20260806 18:39:01.217419 7737 leader_election.cc:288] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:01.217558 7742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:01.217813 7742 raft_consensus.cc:686] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [term 1 LEADER]: Becoming Leader. State: Replica: 709396d630a64cca90171b29673cffd2, State: Running, Role: LEADER
I20260806 18:39:01.218179 7742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709396d630a64cca90171b29673cffd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43949 } }
I20260806 18:39:01.218796 7737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:01.219592 7744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 709396d630a64cca90171b29673cffd2. Latest consensus state: current_term: 1 leader_uuid: "709396d630a64cca90171b29673cffd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709396d630a64cca90171b29673cffd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43949 } } }
I20260806 18:39:01.219738 7744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:01.220041 7743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "709396d630a64cca90171b29673cffd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709396d630a64cca90171b29673cffd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43949 } } }
I20260806 18:39:01.220181 7743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:01.220256 7758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:01.223750 7758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:01.230062 7740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:01.219401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54509 (local address 127.0.0.1:43949)
0806 18:39:01.219522 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:01.219527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:01.220459 (+ 932us) server_negotiation.cc:407] Connection header received
0806 18:39:01.220687 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:01.220691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:01.220751 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:01.220813 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:01.221839 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:01.222597 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:01.223609 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:01.223951 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:01.224052 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:01.226002 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:01.226006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:01.226009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:01.226038 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:01.228015 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:01.228650 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:01.228654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:01.228656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:01.228751 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:01.229133 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:01.229136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:01.229137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:01.229328 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:01.229492 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:01.229784 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 18:39:01.229887 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 18:39:01.230738 7758 catalog_manager.cc:1347] Generated new cluster ID: 0d4dc653f88346bd9e50b8c97509edc7
I20260806 18:39:01.230808 7758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:01.245613 7758 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:01.247027 7758 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:01.253247 7758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 709396d630a64cca90171b29673cffd2: Generated new TSK 0
I20260806 18:39:01.254237 7758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:01.267395 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 7667
W20260806 18:39:01.274554 7650 connection.cc:535] client connection to 127.0.0.1:43949 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43949): Connection reset by peer (error 104)
2026-08-06T18:39:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:01 dist-test-slave-8w9v krb5kdc[7770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:01 dist-test-slave-8w9v krb5kdc[7770](info): set up 2 sockets
Aug 06 18:39:01 dist-test-slave-8w9v krb5kdc[7770](info): commencing operation
krb5kdc: starting...
W20260806 18:39:02.366917 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:02 dist-test-slave-8w9v krb5kdc[7770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:02.508299 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37949
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:02.626180 7784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:02.626487 7784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:02.626566 7784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:02.630851 7784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:02.630930 7784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:02.630950 7784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:02.630968 7784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:02.630985 7784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:02.636979 7784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37949
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:02.638204 7784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:02.639287 7784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:39:02.648660 7784 server_base.cc:1034] running on GCE node
W20260806 18:39:02.648660 7793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:02.648706 7790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:02.648706 7791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:02.649659 7784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:02.650806 7784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:02.651988 7784 hybrid_clock.cc:648] HybridClock initialized: now 1786041542651962 us; error 34 us; skew 500 ppm
Aug 06 18:39:02 dist-test-slave-8w9v krb5kdc[7770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041542, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:02.655045 7784 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:02.656198 7784 webserver.cc:460] Webserver started at http://127.0.0.1:37229/ using document root <none> and password file <none>
I20260806 18:39:02.656793 7784 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:02.656857 7784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:02.657117 7784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:02.659221 7784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "433bf497448b4e9b88783776efbf5178"
format_stamp: "Formatted at 2026-08-06 18:39:02 on dist-test-slave-8w9v"
server_key: "b39240cc86225e9da7821c7339ab3369"
server_key_iv: "26a9dee4a077c8a6d747b0b3d14986ac"
server_key_version: "encryptionkey@0"
I20260806 18:39:02.659822 7784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "433bf497448b4e9b88783776efbf5178"
format_stamp: "Formatted at 2026-08-06 18:39:02 on dist-test-slave-8w9v"
server_key: "b39240cc86225e9da7821c7339ab3369"
server_key_iv: "26a9dee4a077c8a6d747b0b3d14986ac"
server_key_version: "encryptionkey@0"
I20260806 18:39:02.663646 7784 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:39:02.666206 7801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:02.667527 7784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:39:02.667672 7784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "433bf497448b4e9b88783776efbf5178"
format_stamp: "Formatted at 2026-08-06 18:39:02 on dist-test-slave-8w9v"
server_key: "b39240cc86225e9da7821c7339ab3369"
server_key_iv: "26a9dee4a077c8a6d747b0b3d14986ac"
server_key_version: "encryptionkey@0"
I20260806 18:39:02.667789 7784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:02.686581 7784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:02.690107 7784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:02.690349 7784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:02.698051 7784 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37949
I20260806 18:39:02.698055 7857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37949 every 8 connection(s)
I20260806 18:39:02.699159 7784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:02.702245 7858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:02.705999 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 7784
I20260806 18:39:02.706164 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:02.706496 18934 external_mini_cluster.cc:1428] Setting key 99b86ae6ac0874b78da8365913811943
I20260806 18:39:02.709177 7858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178: Bootstrap starting.
I20260806 18:39:02.711885 7858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:02.712782 7858 log.cc:824] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:02.714920 7858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178: No bootstrap required, opened a new log
I20260806 18:39:02.716638 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:02.708153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36117 (local address 127.0.0.1:37949)
0806 18:39:02.708675 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:02.708692 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:02.708731 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:39:02.709499 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:02.709520 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:02.710098 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:02.710513 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:02.710954 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:02.711989 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:02.713024 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:02.713554 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:02.713763 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:02.714757 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:02.714762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:02.714776 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:02.715224 (+ 448us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:02.715283 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:02.715307 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:02.715458 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:02.715653 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 18:39:02.715807 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":155,"threads_started":1}
I20260806 18:39:02.719483 7858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bf497448b4e9b88783776efbf5178" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } }
I20260806 18:39:02.719826 7858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:02.719904 7858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433bf497448b4e9b88783776efbf5178, State: Initialized, Role: FOLLOWER
I20260806 18:39:02.719992 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:02.720425 7858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bf497448b4e9b88783776efbf5178" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } }
I20260806 18:39:02.720561 7858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:02.720650 7858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:02.720773 7858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:02.721935 7858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bf497448b4e9b88783776efbf5178" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } }
I20260806 18:39:02.722378 7858 leader_election.cc:302] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 433bf497448b4e9b88783776efbf5178; no voters:
I20260806 18:39:02.722790 7858 leader_election.cc:288] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:02.722899 7868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:02.723146 7868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [term 1 LEADER]: Becoming Leader. State: Replica: 433bf497448b4e9b88783776efbf5178, State: Running, Role: LEADER
I20260806 18:39:02.723529 7868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bf497448b4e9b88783776efbf5178" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } }
I20260806 18:39:02.724380 7858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:02.725241 7869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "433bf497448b4e9b88783776efbf5178" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bf497448b4e9b88783776efbf5178" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } } }
I20260806 18:39:02.725380 7869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:02.725739 7870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 433bf497448b4e9b88783776efbf5178. Latest consensus state: current_term: 1 leader_uuid: "433bf497448b4e9b88783776efbf5178" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bf497448b4e9b88783776efbf5178" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37949 } } }
I20260806 18:39:02.725834 7870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:02.727813 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:02.722413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39505 (local address 127.0.0.1:37949)
0806 18:39:02.722583 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:02.722589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:02.722719 (+ 130us) server_negotiation.cc:407] Connection header received
0806 18:39:02.722888 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:02.722900 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:02.722957 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:02.723063 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:02.723391 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:02.724175 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:02.725455 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:02.725795 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:02.725918 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:02.727175 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:02.727182 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:02.727188 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:02.727208 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:02.727240 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:02.727254 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:02.727350 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:02.727613 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 18:39:02.727653 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 18:39:02.730037 7883 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:02.730147 7883 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:02.730233 7877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:02.731062 7877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:02.737722 7877 catalog_manager.cc:1347] Generated new cluster ID: 0988f53d6b14401eb44767dbd40723a0
I20260806 18:39:02.737819 7877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:02.750962 7877 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:02.752065 7877 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:02.762970 7877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 433bf497448b4e9b88783776efbf5178: Generated new TSK 0
I20260806 18:39:02.763918 7877 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:02.774075 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 7784
W20260806 18:39:02.780957 7768 connection.cc:535] client connection to 127.0.0.1:37949 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37949): Connection reset by peer (error 104)
2026-08-06T18:39:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:02 dist-test-slave-8w9v krb5kdc[7889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:02 dist-test-slave-8w9v krb5kdc[7889](info): set up 2 sockets
Aug 06 18:39:02 dist-test-slave-8w9v krb5kdc[7889](info): commencing operation
krb5kdc: starting...
W20260806 18:39:03.892436 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:03 dist-test-slave-8w9v krb5kdc[7889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:04.036293 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38291
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:04.157506 7904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:04.157861 7904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:04.157944 7904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:04.162720 7904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:04.162820 7904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:04.162856 7904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:04.162902 7904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:04.162940 7904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:04.168608 7904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38291
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:04.169730 7904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:04.170753 7904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:04.179549 7910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:04.179534 7909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:04.179750 7904 server_base.cc:1034] running on GCE node
W20260806 18:39:04.179563 7912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:04.180505 7904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:04.181614 7904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:04.182821 7904 hybrid_clock.cc:648] HybridClock initialized: now 1786041544182782 us; error 43 us; skew 500 ppm
Aug 06 18:39:04 dist-test-slave-8w9v krb5kdc[7889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041544, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:04.191563 7904 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:04.192679 7904 webserver.cc:460] Webserver started at http://127.0.0.1:40535/ using document root <none> and password file <none>
I20260806 18:39:04.193298 7904 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:04.193396 7904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:04.193645 7904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:04.195799 7904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "844ac945645948f08d149d3b9ab3cf43"
format_stamp: "Formatted at 2026-08-06 18:39:04 on dist-test-slave-8w9v"
server_key: "b766400e9a239ed32fa30d75c59b4279"
server_key_iv: "8c18867abacebe9b0d1a27c3413a1d24"
server_key_version: "encryptionkey@0"
I20260806 18:39:04.196426 7904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "844ac945645948f08d149d3b9ab3cf43"
format_stamp: "Formatted at 2026-08-06 18:39:04 on dist-test-slave-8w9v"
server_key: "b766400e9a239ed32fa30d75c59b4279"
server_key_iv: "8c18867abacebe9b0d1a27c3413a1d24"
server_key_version: "encryptionkey@0"
I20260806 18:39:04.201486 7904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 18:39:04.204236 7919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:04.205621 7904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:04.205783 7904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "844ac945645948f08d149d3b9ab3cf43"
format_stamp: "Formatted at 2026-08-06 18:39:04 on dist-test-slave-8w9v"
server_key: "b766400e9a239ed32fa30d75c59b4279"
server_key_iv: "8c18867abacebe9b0d1a27c3413a1d24"
server_key_version: "encryptionkey@0"
I20260806 18:39:04.205924 7904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:04.211524 7904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:04.214973 7904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:04.215188 7904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:04.222808 7904 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38291
I20260806 18:39:04.222860 7974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38291 every 8 connection(s)
I20260806 18:39:04.223783 7904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:04.224167 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 7904
I20260806 18:39:04.224325 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:04.224644 18934 external_mini_cluster.cc:1428] Setting key 9d4c6a24b009b4f90589275fefb16853
I20260806 18:39:04.227272 7975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:04.231081 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:04.226121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48745 (local address 127.0.0.1:38291)
0806 18:39:04.226558 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:04.226578 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:04.226622 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:39:04.227525 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:04.227549 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:04.227918 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:04.228702 (+ 784us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:04.229463 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:04.229470 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:04.229485 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:04.229500 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:04.229561 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:04.229589 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:04.229761 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:04.229920 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 18:39:04.230082 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":115,"threads_started":1}
I20260806 18:39:04.234515 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:04.235430 7975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43: Bootstrap starting.
I20260806 18:39:04.238456 7975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:04.239297 7978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:04.237314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58569 (local address 127.0.0.1:38291)
0806 18:39:04.237483 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:04.237488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:04.237669 (+ 181us) server_negotiation.cc:407] Connection header received
0806 18:39:04.237857 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:04.237861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:04.237907 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:04.238005 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:04.238864 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:04.238867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:04.238870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:04.238883 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:04.238912 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:04.238932 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:04.239046 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:04.239123 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 18:39:04.239168 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:39:04.239663 7975 log.cc:824] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:04.242388 7975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43: No bootstrap required, opened a new log
I20260806 18:39:04.246666 7975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844ac945645948f08d149d3b9ab3cf43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38291 } }
I20260806 18:39:04.246955 7975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:04.247069 7975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 844ac945645948f08d149d3b9ab3cf43, State: Initialized, Role: FOLLOWER
I20260806 18:39:04.247586 7975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844ac945645948f08d149d3b9ab3cf43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38291 } }
I20260806 18:39:04.247792 7975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:04.247869 7975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:04.248019 7975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:04.249168 7975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844ac945645948f08d149d3b9ab3cf43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38291 } }
I20260806 18:39:04.249599 7975 leader_election.cc:302] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 844ac945645948f08d149d3b9ab3cf43; no voters:
I20260806 18:39:04.250013 7975 leader_election.cc:288] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:04.250341 7985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:04.250609 7985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [term 1 LEADER]: Becoming Leader. State: Replica: 844ac945645948f08d149d3b9ab3cf43, State: Running, Role: LEADER
I20260806 18:39:04.250996 7985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844ac945645948f08d149d3b9ab3cf43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38291 } }
I20260806 18:39:04.251534 7975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:04.252842 7988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 844ac945645948f08d149d3b9ab3cf43. Latest consensus state: current_term: 1 leader_uuid: "844ac945645948f08d149d3b9ab3cf43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844ac945645948f08d149d3b9ab3cf43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38291 } } }
I20260806 18:39:04.252995 7988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:04.253302 7987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "844ac945645948f08d149d3b9ab3cf43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844ac945645948f08d149d3b9ab3cf43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38291 } } }
I20260806 18:39:04.253412 7995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:04.253515 7987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:04.257076 7995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:04.263863 7995 catalog_manager.cc:1347] Generated new cluster ID: 2a3817aca03d42d8b5b862caac956fb8
I20260806 18:39:04.263965 7995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:04.293219 7995 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:04.293593 7995 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:04.303783 7995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 844ac945645948f08d149d3b9ab3cf43: Generated new TSK 0
I20260806 18:39:04.305419 7995 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:04.322651 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 7904
2026-08-06T18:39:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:04 dist-test-slave-8w9v krb5kdc[8008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:04 dist-test-slave-8w9v krb5kdc[8008](info): set up 2 sockets
Aug 06 18:39:04 dist-test-slave-8w9v krb5kdc[8008](info): commencing operation
krb5kdc: starting...
W20260806 18:39:05.403704 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:05 dist-test-slave-8w9v krb5kdc[8008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:05.539362 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43637
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:05.659758 8021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:05.660120 8021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:05.660256 8021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:05.664434 8021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:05.664552 8021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:05.664594 8021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:05.664634 8021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:05.664667 8021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:05.670469 8021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:05.671684 8021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:05.673286 8021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:05.683662 8027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:05.683676 8030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:05.683718 8028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:05.683940 8021 server_base.cc:1034] running on GCE node
I20260806 18:39:05.684660 8021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:05.685894 8021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:05.687112 8021 hybrid_clock.cc:648] HybridClock initialized: now 1786041545687091 us; error 36 us; skew 500 ppm
Aug 06 18:39:05 dist-test-slave-8w9v krb5kdc[8008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041545, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:05.690109 8021 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:05.691133 8021 webserver.cc:460] Webserver started at http://127.0.0.1:33769/ using document root <none> and password file <none>
I20260806 18:39:05.691694 8021 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:05.691753 8021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:05.691938 8021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:05.694286 8021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "1ef9028a72fb4c059890a45e3e73e68e"
format_stamp: "Formatted at 2026-08-06 18:39:05 on dist-test-slave-8w9v"
server_key: "8a83f9b0578fab6ffec3616e5568db91"
server_key_iv: "2614d822b9a92f84343a216f8d47c79f"
server_key_version: "encryptionkey@0"
I20260806 18:39:05.694866 8021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "1ef9028a72fb4c059890a45e3e73e68e"
format_stamp: "Formatted at 2026-08-06 18:39:05 on dist-test-slave-8w9v"
server_key: "8a83f9b0578fab6ffec3616e5568db91"
server_key_iv: "2614d822b9a92f84343a216f8d47c79f"
server_key_version: "encryptionkey@0"
I20260806 18:39:05.699895 8021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 18:39:05.703891 8037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:05.705542 8021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 18:39:05.705734 8021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "1ef9028a72fb4c059890a45e3e73e68e"
format_stamp: "Formatted at 2026-08-06 18:39:05 on dist-test-slave-8w9v"
server_key: "8a83f9b0578fab6ffec3616e5568db91"
server_key_iv: "2614d822b9a92f84343a216f8d47c79f"
server_key_version: "encryptionkey@0"
I20260806 18:39:05.705878 8021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:05.726017 8021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:05.729460 8021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:05.729687 8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:05.737437 8021 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43637
I20260806 18:39:05.737457 8090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43637 every 8 connection(s)
I20260806 18:39:05.738495 8021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:05.741529 8091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:05.747395 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 8021
I20260806 18:39:05.747541 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:05.747841 18934 external_mini_cluster.cc:1428] Setting key a0a9d39a7da58145d4e94b447f42f1bb
I20260806 18:39:05.749131 8091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e: Bootstrap starting.
I20260806 18:39:05.753193 8091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:05.754189 8091 log.cc:824] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:05.760529 8091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e: No bootstrap required, opened a new log
I20260806 18:39:05.765515 8091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef9028a72fb4c059890a45e3e73e68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } }
I20260806 18:39:05.765889 8091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:05.765959 8091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef9028a72fb4c059890a45e3e73e68e, State: Initialized, Role: FOLLOWER
I20260806 18:39:05.766603 8091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef9028a72fb4c059890a45e3e73e68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } }
I20260806 18:39:05.766803 8091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:05.766883 8091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:05.766995 8091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [term 0 FOLLOWER]: Advancing to term 1
Aug 06 18:39:05 dist-test-slave-8w9v krb5kdc[8008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:39:05.768308 8091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef9028a72fb4c059890a45e3e73e68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } }
I20260806 18:39:05.768718 8091 leader_election.cc:302] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef9028a72fb4c059890a45e3e73e68e; no voters:
I20260806 18:39:05.769129 8091 leader_election.cc:288] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:05.769243 8096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:05.769431 8096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [term 1 LEADER]: Becoming Leader. State: Replica: 1ef9028a72fb4c059890a45e3e73e68e, State: Running, Role: LEADER
I20260806 18:39:05.769760 8096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef9028a72fb4c059890a45e3e73e68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } }
I20260806 18:39:05.770651 8091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:05.771143 8098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ef9028a72fb4c059890a45e3e73e68e. Latest consensus state: current_term: 1 leader_uuid: "1ef9028a72fb4c059890a45e3e73e68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef9028a72fb4c059890a45e3e73e68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } } }
I20260806 18:39:05.771327 8098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:05.771571 8097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ef9028a72fb4c059890a45e3e73e68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef9028a72fb4c059890a45e3e73e68e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43637 } } }
I20260806 18:39:05.771699 8097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:05.771943 8102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:05.775478 8094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:05.753802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57249 (local address 127.0.0.1:43637)
0806 18:39:05.758487 (+ 4685us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:05.758513 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:05.758572 (+ 59us) server_negotiation.cc:407] Connection header received
0806 18:39:05.759692 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:05.759723 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:05.760276 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:05.760707 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:05.761853 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:05.763235 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:05.764768 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:05.765354 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:05.765499 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:05.768750 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:05.768756 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:05.768771 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:05.768804 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:05.771161 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:05.771834 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:05.771842 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:05.771848 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:05.771996 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:05.772423 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:05.772426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:05.772428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:05.772974 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:05.773965 (+ 991us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:05.774128 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 18:39:05.774474 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4449,"thread_start_us":177,"threads_started":1}
I20260806 18:39:05.775655 8102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:05.779261 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:05.782455 8102 catalog_manager.cc:1347] Generated new cluster ID: d14c604b75c7418d88e9088874887f72
I20260806 18:39:05.782541 8102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:05.794109 8094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:05.783070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36141 (local address 127.0.0.1:43637)
0806 18:39:05.783204 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:05.783209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:05.784508 (+ 1299us) server_negotiation.cc:407] Connection header received
0806 18:39:05.784645 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:05.784649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:05.784716 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:05.784826 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:05.785679 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:05.786565 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:05.787784 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:05.788309 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:05.788438 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:05.790167 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:05.790171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:05.790174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:05.790204 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:05.791982 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:05.792507 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:05.792511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:05.792513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:05.792627 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:05.792980 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:05.792984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:05.792986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:05.793214 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:05.793341 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:05.793612 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 18:39:05.793742 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":49}
I20260806 18:39:05.802790 8102 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:05.803797 8102 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:05.812206 8102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ef9028a72fb4c059890a45e3e73e68e: Generated new TSK 0
I20260806 18:39:05.813184 8102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:05.824317 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 8021
2026-08-06T18:39:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:05 dist-test-slave-8w9v krb5kdc[8125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:05 dist-test-slave-8w9v krb5kdc[8125](info): set up 2 sockets
Aug 06 18:39:05 dist-test-slave-8w9v krb5kdc[8125](info): commencing operation
krb5kdc: starting...
W20260806 18:39:06.898270 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:06 dist-test-slave-8w9v krb5kdc[8125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:07.036615 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37827
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:07.157230 8139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:07.157569 8139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:07.157620 8139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:07.161932 8139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:07.162031 8139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:07.162055 8139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:07.162075 8139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:07.162094 8139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:07.167878 8139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:07.169180 8139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:07.170347 8139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:07.180064 8146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:07.180064 8145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:07.180078 8148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:07.180179 8139 server_base.cc:1034] running on GCE node
I20260806 18:39:07.181102 8139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:07.182411 8139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:07.183619 8139 hybrid_clock.cc:648] HybridClock initialized: now 1786041547183598 us; error 37 us; skew 500 ppm
Aug 06 18:39:07 dist-test-slave-8w9v krb5kdc[8125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041547, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:07.186790 8139 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:07.187799 8139 webserver.cc:460] Webserver started at http://127.0.0.1:43729/ using document root <none> and password file <none>
I20260806 18:39:07.188467 8139 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:07.188539 8139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:07.188726 8139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:07.190955 8139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "b3416401fdc2495787f138502bd4fe38"
format_stamp: "Formatted at 2026-08-06 18:39:07 on dist-test-slave-8w9v"
server_key: "9f7e46fd4ac5e6e668a453999306331c"
server_key_iv: "b388b121d5c263268437ae989ee93cd2"
server_key_version: "encryptionkey@0"
I20260806 18:39:07.191516 8139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "b3416401fdc2495787f138502bd4fe38"
format_stamp: "Formatted at 2026-08-06 18:39:07 on dist-test-slave-8w9v"
server_key: "9f7e46fd4ac5e6e668a453999306331c"
server_key_iv: "b388b121d5c263268437ae989ee93cd2"
server_key_version: "encryptionkey@0"
I20260806 18:39:07.195714 8139 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:39:07.198647 8156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:07.200322 8139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 18:39:07.200484 8139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "b3416401fdc2495787f138502bd4fe38"
format_stamp: "Formatted at 2026-08-06 18:39:07 on dist-test-slave-8w9v"
server_key: "9f7e46fd4ac5e6e668a453999306331c"
server_key_iv: "b388b121d5c263268437ae989ee93cd2"
server_key_version: "encryptionkey@0"
I20260806 18:39:07.200618 8139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:07.244961 8139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:07.248742 8139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:07.249082 8139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:07.260263 8139 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37827
I20260806 18:39:07.260444 8208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37827 every 8 connection(s)
I20260806 18:39:07.261641 8139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:07.265246 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 8139
I20260806 18:39:07.265339 8209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:07.265421 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:07.265774 18934 external_mini_cluster.cc:1428] Setting key b5546cd760efcccc428e79b3b92c1936
I20260806 18:39:07.273732 8209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38: Bootstrap starting.
I20260806 18:39:07.275935 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:07.267337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36597 (local address 127.0.0.1:37827)
0806 18:39:07.267817 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:07.267834 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:07.267871 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:39:07.268618 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:07.268655 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:07.269015 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:07.269300 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:07.269774 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:07.270803 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:07.272115 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:07.272725 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:07.272866 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:07.273969 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:07.273983 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:07.274010 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:07.274462 (+ 452us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:07.274527 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:07.274553 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:07.274637 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:07.274846 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 18:39:07.275095 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":162,"threads_started":1}
I20260806 18:39:07.276666 8209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:07.277588 8209 log.cc:824] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:07.279533 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:07.280093 8209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38: No bootstrap required, opened a new log
I20260806 18:39:07.284714 8209 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3416401fdc2495787f138502bd4fe38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37827 } }
I20260806 18:39:07.285051 8209 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:07.285113 8209 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3416401fdc2495787f138502bd4fe38, State: Initialized, Role: FOLLOWER
I20260806 18:39:07.285696 8209 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3416401fdc2495787f138502bd4fe38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37827 } }
I20260806 18:39:07.285907 8209 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:07.286000 8209 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:07.286118 8209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:07.286618 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:07.282010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41415 (local address 127.0.0.1:37827)
0806 18:39:07.282154 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:07.282160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:07.282395 (+ 235us) server_negotiation.cc:407] Connection header received
0806 18:39:07.282572 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:07.282577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:07.282631 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:07.282745 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:07.283073 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:07.283616 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:07.284611 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:07.284867 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:07.285002 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:07.286035 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:07.286041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:07.286045 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:07.286065 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:07.286099 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:07.286115 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:07.286213 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:07.286405 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 18:39:07.286480 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:39:07.287367 8209 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3416401fdc2495787f138502bd4fe38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37827 } }
I20260806 18:39:07.287806 8209 leader_election.cc:302] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3416401fdc2495787f138502bd4fe38; no voters:
I20260806 18:39:07.288278 8209 leader_election.cc:288] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:07.288409 8220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:07.288684 8220 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [term 1 LEADER]: Becoming Leader. State: Replica: b3416401fdc2495787f138502bd4fe38, State: Running, Role: LEADER
I20260806 18:39:07.289048 8220 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3416401fdc2495787f138502bd4fe38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37827 } }
I20260806 18:39:07.289662 8209 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:07.290963 8222 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3416401fdc2495787f138502bd4fe38. Latest consensus state: current_term: 1 leader_uuid: "b3416401fdc2495787f138502bd4fe38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3416401fdc2495787f138502bd4fe38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37827 } } }
I20260806 18:39:07.291112 8222 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:07.291437 8221 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3416401fdc2495787f138502bd4fe38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3416401fdc2495787f138502bd4fe38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37827 } } }
I20260806 18:39:07.291522 8230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:07.291563 8221 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:07.295327 8230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:07.302395 8230 catalog_manager.cc:1347] Generated new cluster ID: e2e427086fb94873ab079c5dd688c78c
I20260806 18:39:07.302500 8230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:07.313405 8230 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:07.314850 8230 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:07.320830 8230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3416401fdc2495787f138502bd4fe38: Generated new TSK 0
I20260806 18:39:07.321631 8230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:07.337481 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 8139
2026-08-06T18:39:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:07 dist-test-slave-8w9v krb5kdc[8244](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:07 dist-test-slave-8w9v krb5kdc[8244](info): set up 2 sockets
Aug 06 18:39:07 dist-test-slave-8w9v krb5kdc[8244](info): commencing operation
krb5kdc: starting...
W20260806 18:39:08.418354 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:08 dist-test-slave-8w9v krb5kdc[8244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:08.555905 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45473
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:08.675419 8257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:08.675743 8257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:08.675788 8257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:08.680917 8257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:08.681016 8257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:08.681046 8257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:08.681066 8257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:08.681082 8257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:08.686973 8257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:08.688256 8257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:08.689334 8257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:08.698323 8264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:08.698289 8263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:08.698346 8266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:08.698895 8257 server_base.cc:1034] running on GCE node
I20260806 18:39:08.699771 8257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:08.701192 8257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:08.702426 8257 hybrid_clock.cc:648] HybridClock initialized: now 1786041548702409 us; error 37 us; skew 500 ppm
Aug 06 18:39:08 dist-test-slave-8w9v krb5kdc[8244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041548, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:08.705780 8257 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:08.706900 8257 webserver.cc:460] Webserver started at http://127.0.0.1:35663/ using document root <none> and password file <none>
I20260806 18:39:08.707515 8257 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:08.707585 8257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:08.707783 8257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:08.710156 8257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "490d9a1e280e48288298d7c2fc65aed6"
format_stamp: "Formatted at 2026-08-06 18:39:08 on dist-test-slave-8w9v"
server_key: "2464867fed786b908e1fcb8bde8a8df4"
server_key_iv: "5c3ae4c3aabb9cf547a34c2a0e587d72"
server_key_version: "encryptionkey@0"
I20260806 18:39:08.710768 8257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "490d9a1e280e48288298d7c2fc65aed6"
format_stamp: "Formatted at 2026-08-06 18:39:08 on dist-test-slave-8w9v"
server_key: "2464867fed786b908e1fcb8bde8a8df4"
server_key_iv: "5c3ae4c3aabb9cf547a34c2a0e587d72"
server_key_version: "encryptionkey@0"
I20260806 18:39:08.715041 8257 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:39:08.717854 8273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:08.719192 8257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 18:39:08.719313 8257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "490d9a1e280e48288298d7c2fc65aed6"
format_stamp: "Formatted at 2026-08-06 18:39:08 on dist-test-slave-8w9v"
server_key: "2464867fed786b908e1fcb8bde8a8df4"
server_key_iv: "5c3ae4c3aabb9cf547a34c2a0e587d72"
server_key_version: "encryptionkey@0"
I20260806 18:39:08.719422 8257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:08.724339 8257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:08.727634 8257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:08.727802 8257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:08.735312 8257 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45473
I20260806 18:39:08.735342 8326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45473 every 8 connection(s)
I20260806 18:39:08.736356 8257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:08.739385 8327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:08.743871 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 8257
I20260806 18:39:08.744045 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:08.744398 18934 external_mini_cluster.cc:1428] Setting key 0e4eac55c75241baa435e1a1f4a0a7de
I20260806 18:39:08.746425 8327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6: Bootstrap starting.
I20260806 18:39:08.749171 8327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:08.750146 8327 log.cc:824] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:08.750656 8330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:08.746177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56641 (local address 127.0.0.1:45473)
0806 18:39:08.746758 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:08.746774 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:08.746813 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:39:08.747518 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:08.747537 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:08.747848 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:08.748270 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:08.749238 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:08.749247 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:08.749265 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:08.749282 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:08.749358 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:08.749381 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:08.749517 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:08.749625 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 18:39:08.749824 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":167,"threads_started":1}
I20260806 18:39:08.752406 8327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6: No bootstrap required, opened a new log
I20260806 18:39:08.754030 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:08.757633 8327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490d9a1e280e48288298d7c2fc65aed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } }
I20260806 18:39:08.758023 8327 raft_consensus.cc:374] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:08.758118 8327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490d9a1e280e48288298d7c2fc65aed6, State: Initialized, Role: FOLLOWER
I20260806 18:39:08.758567 8330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:08.756381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40251 (local address 127.0.0.1:45473)
0806 18:39:08.756523 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:08.756527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:08.756822 (+ 295us) server_negotiation.cc:407] Connection header received
0806 18:39:08.756947 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:08.756950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:08.756988 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:08.757068 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:08.757941 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:08.757944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:08.757946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:08.757960 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:08.758047 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:08.758067 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:08.758182 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:08.758355 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 18:39:08.758405 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:39:08.758692 8327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490d9a1e280e48288298d7c2fc65aed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } }
I20260806 18:39:08.758865 8327 raft_consensus.cc:388] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:08.758942 8327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:08.759052 8327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:08.760319 8327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490d9a1e280e48288298d7c2fc65aed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } }
I20260806 18:39:08.760701 8327 leader_election.cc:302] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 490d9a1e280e48288298d7c2fc65aed6; no voters:
I20260806 18:39:08.761046 8327 leader_election.cc:288] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:08.761225 8337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:08.761507 8337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [term 1 LEADER]: Becoming Leader. State: Replica: 490d9a1e280e48288298d7c2fc65aed6, State: Running, Role: LEADER
I20260806 18:39:08.761875 8337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490d9a1e280e48288298d7c2fc65aed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } }
I20260806 18:39:08.762527 8327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:08.763885 8339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 490d9a1e280e48288298d7c2fc65aed6. Latest consensus state: current_term: 1 leader_uuid: "490d9a1e280e48288298d7c2fc65aed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490d9a1e280e48288298d7c2fc65aed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } } }
I20260806 18:39:08.764043 8339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:08.764454 8338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "490d9a1e280e48288298d7c2fc65aed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490d9a1e280e48288298d7c2fc65aed6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45473 } } }
I20260806 18:39:08.764555 8338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6 [sys.catalog]: This master's current role is: LEADER
W20260806 18:39:08.768095 8353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:08.768235 8353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:08.768308 8346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:08.769045 8346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:08.775537 8346 catalog_manager.cc:1347] Generated new cluster ID: ff4e66e62a91472ca56671a030bd98a3
I20260806 18:39:08.775619 8346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:08.816098 8346 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:08.816545 8346 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:08.824873 8346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 490d9a1e280e48288298d7c2fc65aed6: Generated new TSK 0
I20260806 18:39:08.825891 8346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:08.833065 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 8257
2026-08-06T18:39:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:08 dist-test-slave-8w9v krb5kdc[8358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:08 dist-test-slave-8w9v krb5kdc[8358](info): set up 2 sockets
Aug 06 18:39:08 dist-test-slave-8w9v krb5kdc[8358](info): commencing operation
krb5kdc: starting...
W20260806 18:39:09.943657 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:10 dist-test-slave-8w9v krb5kdc[8358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:10.088732 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38633
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:10.202930 8373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:10.203318 8373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:10.203408 8373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:10.207839 8373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:10.207952 8373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:10.207993 8373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:10.208031 8373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:10.208067 8373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:10.213656 8373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:10.214884 8373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:10.215983 8373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:10.224818 8381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:10.224792 8378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:10.224819 8379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:10.224982 8373 server_base.cc:1034] running on GCE node
I20260806 18:39:10.225694 8373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:10.226785 8373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:10.228080 8373 hybrid_clock.cc:648] HybridClock initialized: now 1786041550228071 us; error 45 us; skew 500 ppm
Aug 06 18:39:10 dist-test-slave-8w9v krb5kdc[8358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041550, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:10.236347 8373 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:10.237519 8373 webserver.cc:460] Webserver started at http://127.0.0.1:39769/ using document root <none> and password file <none>
I20260806 18:39:10.238240 8373 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:10.238363 8373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:10.238617 8373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:10.240885 8373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "7a3f7a1135424321a060a5c64091857c"
format_stamp: "Formatted at 2026-08-06 18:39:10 on dist-test-slave-8w9v"
server_key: "c7379ac89a64db16d5f4b33cca932243"
server_key_iv: "7c69985787a2bf740d1ad81bb77b116e"
server_key_version: "encryptionkey@0"
I20260806 18:39:10.241496 8373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "7a3f7a1135424321a060a5c64091857c"
format_stamp: "Formatted at 2026-08-06 18:39:10 on dist-test-slave-8w9v"
server_key: "c7379ac89a64db16d5f4b33cca932243"
server_key_iv: "7c69985787a2bf740d1ad81bb77b116e"
server_key_version: "encryptionkey@0"
I20260806 18:39:10.245559 8373 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:39:10.248430 8388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:10.249842 8373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:39:10.250020 8373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "7a3f7a1135424321a060a5c64091857c"
format_stamp: "Formatted at 2026-08-06 18:39:10 on dist-test-slave-8w9v"
server_key: "c7379ac89a64db16d5f4b33cca932243"
server_key_iv: "7c69985787a2bf740d1ad81bb77b116e"
server_key_version: "encryptionkey@0"
I20260806 18:39:10.250154 8373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:10.270026 8373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:10.273802 8373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:10.274039 8373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:10.282086 8373 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38633
I20260806 18:39:10.282140 8444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38633 every 8 connection(s)
I20260806 18:39:10.283309 8373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:10.286581 8445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:10.286665 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 8373
I20260806 18:39:10.286780 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:10.287101 18934 external_mini_cluster.cc:1428] Setting key ed1db0e2b04ef13cffde9916e0b90869
I20260806 18:39:10.294958 8445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c: Bootstrap starting.
Aug 06 18:39:10 dist-test-slave-8w9v krb5kdc[8358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:39:10.297927 8445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:10.298719 8445 log.cc:824] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:10.300896 8445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c: No bootstrap required, opened a new log
I20260806 18:39:10.302968 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:10.288606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:54057 (local address 127.0.0.1:38633)
0806 18:39:10.289063 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:10.289078 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:10.289115 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:39:10.289984 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:10.290003 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:10.290356 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:10.290638 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:10.291639 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:10.292788 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:10.294002 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:10.294484 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:10.294613 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:10.297709 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:10.297715 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:10.297730 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:10.297775 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:10.299744 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:10.300433 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:10.300438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:10.300443 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:10.300556 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:10.300944 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:10.300947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:10.300948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:10.301400 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:10.301631 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:10.301977 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 18:39:10.302203 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":158,"threads_started":1}
I20260806 18:39:10.304970 8445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3f7a1135424321a060a5c64091857c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } }
I20260806 18:39:10.305307 8445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:10.305384 8445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a3f7a1135424321a060a5c64091857c, State: Initialized, Role: FOLLOWER
I20260806 18:39:10.305855 8445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3f7a1135424321a060a5c64091857c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } }
I20260806 18:39:10.305992 8445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:10.306073 8445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:10.306213 8445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:10.306324 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:10.307670 8445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3f7a1135424321a060a5c64091857c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } }
I20260806 18:39:10.308107 8445 leader_election.cc:302] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a3f7a1135424321a060a5c64091857c; no voters:
I20260806 18:39:10.308518 8445 leader_election.cc:288] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:10.308650 8455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:10.308938 8455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [term 1 LEADER]: Becoming Leader. State: Replica: 7a3f7a1135424321a060a5c64091857c, State: Running, Role: LEADER
I20260806 18:39:10.309278 8455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3f7a1135424321a060a5c64091857c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } }
I20260806 18:39:10.310319 8445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:10.311364 8457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a3f7a1135424321a060a5c64091857c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3f7a1135424321a060a5c64091857c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } } }
I20260806 18:39:10.311522 8457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:10.311800 8456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a3f7a1135424321a060a5c64091857c. Latest consensus state: current_term: 1 leader_uuid: "7a3f7a1135424321a060a5c64091857c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3f7a1135424321a060a5c64091857c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } } }
I20260806 18:39:10.311913 8456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c [sys.catalog]: This master's current role is: LEADER
W20260806 18:39:10.317176 8471 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:10.317281 8471 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:10.317391 8465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:10.318282 8465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:10.321147 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:10.308906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:46177 (local address 127.0.0.1:38633)
0806 18:39:10.309036 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:10.309042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:10.309433 (+ 391us) server_negotiation.cc:407] Connection header received
0806 18:39:10.309866 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:10.309869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:10.309928 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:10.310015 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:10.311204 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:10.312075 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:10.313524 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:10.313858 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:10.313941 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:10.316416 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:10.316420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:10.316423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:10.316464 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:10.318468 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:10.319160 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:10.319164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:10.319166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:10.319273 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:10.319700 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:10.319705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:10.319707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:10.319978 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:10.320176 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:10.320583 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 18:39:10.320973 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:39:10.324903 8465 catalog_manager.cc:1347] Generated new cluster ID: 8604db161c764ac9b2335aa9bf2f9878
I20260806 18:39:10.324985 8465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:10.331687 8465 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:10.332926 8465 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:10.338984 8465 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a3f7a1135424321a060a5c64091857c: Generated new TSK 0
I20260806 18:39:10.339861 8465 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:10.351321 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 8373
W20260806 18:39:10.358357 8356 connection.cc:535] client connection to 127.0.0.1:38633 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38633): Connection reset by peer (error 104)
2026-08-06T18:39:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:10 dist-test-slave-8w9v krb5kdc[8477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:10 dist-test-slave-8w9v krb5kdc[8477](info): set up 2 sockets
Aug 06 18:39:10 dist-test-slave-8w9v krb5kdc[8477](info): commencing operation
krb5kdc: starting...
W20260806 18:39:11.447467 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:11 dist-test-slave-8w9v krb5kdc[8477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:11.588559 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46403
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:11.709802 8491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:11.710155 8491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:11.710239 8491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:11.714432 8491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:11.714551 8491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:11.714593 8491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:11.714655 8491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:11.714695 8491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:11.720393 8491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:11.721603 8491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:11.722694 8491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:39:11.731994 8491 server_base.cc:1034] running on GCE node
W20260806 18:39:11.731966 8496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:11.731999 8500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:11.732043 8497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:11.732901 8491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:11.734126 8491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:11.735309 8491 hybrid_clock.cc:648] HybridClock initialized: now 1786041551735289 us; error 36 us; skew 500 ppm
Aug 06 18:39:11 dist-test-slave-8w9v krb5kdc[8477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041551, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:11.738385 8491 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:11.739375 8491 webserver.cc:460] Webserver started at http://127.0.0.1:41487/ using document root <none> and password file <none>
I20260806 18:39:11.739945 8491 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:11.740012 8491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:11.740298 8491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:11.742564 8491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "b54c84ea3a604b4798522c413d7ca749"
format_stamp: "Formatted at 2026-08-06 18:39:11 on dist-test-slave-8w9v"
server_key: "afce1b5bf9e64401dd1358a8a14eafee"
server_key_iv: "4c8f301dcddd6ee73028ac35688c740d"
server_key_version: "encryptionkey@0"
I20260806 18:39:11.743100 8491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "b54c84ea3a604b4798522c413d7ca749"
format_stamp: "Formatted at 2026-08-06 18:39:11 on dist-test-slave-8w9v"
server_key: "afce1b5bf9e64401dd1358a8a14eafee"
server_key_iv: "4c8f301dcddd6ee73028ac35688c740d"
server_key_version: "encryptionkey@0"
I20260806 18:39:11.748865 8491 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 18:39:11.752954 8508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:11.754582 8491 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:39:11.754812 8491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "b54c84ea3a604b4798522c413d7ca749"
format_stamp: "Formatted at 2026-08-06 18:39:11 on dist-test-slave-8w9v"
server_key: "afce1b5bf9e64401dd1358a8a14eafee"
server_key_iv: "4c8f301dcddd6ee73028ac35688c740d"
server_key_version: "encryptionkey@0"
I20260806 18:39:11.754973 8491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:11.783058 8491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:11.786679 8491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:11.786955 8491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:11.794932 8491 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46403
I20260806 18:39:11.795014 8563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46403 every 8 connection(s)
I20260806 18:39:11.796036 8491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:11.797354 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 8491
I20260806 18:39:11.797533 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:11.797827 18934 external_mini_cluster.cc:1428] Setting key 85e43171d3cc6e2bf73972828b6485c4
I20260806 18:39:11.799363 8564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:11.806406 8564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749: Bootstrap starting.
I20260806 18:39:11.808694 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:11.799332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:50759 (local address 127.0.0.1:46403)
0806 18:39:11.799807 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:11.799825 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:11.799860 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:39:11.800604 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:11.800622 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:11.800976 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:11.801266 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:11.801677 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:11.802701 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:11.803948 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:11.804573 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:11.804714 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:11.805939 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:11.805948 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:11.805970 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:11.806847 (+ 877us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:11.806958 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:11.806996 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:11.807367 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:11.807554 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 18:39:11.807764 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":150,"threads_started":1}
I20260806 18:39:11.809351 8564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:11.810276 8564 log.cc:824] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:11.812533 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:11.812994 8564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749: No bootstrap required, opened a new log
I20260806 18:39:11.817490 8564 raft_consensus.cc:348] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54c84ea3a604b4798522c413d7ca749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46403 } }
I20260806 18:39:11.817806 8564 raft_consensus.cc:374] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:11.817878 8564 raft_consensus.cc:729] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54c84ea3a604b4798522c413d7ca749, State: Initialized, Role: FOLLOWER
I20260806 18:39:11.818434 8564 consensus_queue.cc:260] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54c84ea3a604b4798522c413d7ca749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46403 } }
I20260806 18:39:11.818639 8564 raft_consensus.cc:388] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:11.818748 8564 raft_consensus.cc:482] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:11.818881 8564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:11.819377 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:11.814772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:34703 (local address 127.0.0.1:46403)
0806 18:39:11.814908 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:11.814913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:11.815137 (+ 224us) server_negotiation.cc:407] Connection header received
0806 18:39:11.815319 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:11.815322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:11.815371 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:11.815521 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:11.815812 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:11.816457 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:11.817370 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:11.817611 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:11.817731 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:11.818781 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:11.818786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:11.818789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:11.818812 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:11.818845 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:11.818861 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:11.818993 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:11.819188 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 18:39:11.819234 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:39:11.820185 8564 raft_consensus.cc:504] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54c84ea3a604b4798522c413d7ca749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46403 } }
I20260806 18:39:11.820638 8564 leader_election.cc:302] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b54c84ea3a604b4798522c413d7ca749; no voters:
I20260806 18:39:11.821031 8564 leader_election.cc:288] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:11.821118 8575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:11.821399 8575 raft_consensus.cc:686] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [term 1 LEADER]: Becoming Leader. State: Replica: b54c84ea3a604b4798522c413d7ca749, State: Running, Role: LEADER
I20260806 18:39:11.821753 8575 consensus_queue.cc:237] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54c84ea3a604b4798522c413d7ca749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46403 } }
I20260806 18:39:11.822490 8564 sys_catalog.cc:564] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:11.823575 8576 sys_catalog.cc:455] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b54c84ea3a604b4798522c413d7ca749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54c84ea3a604b4798522c413d7ca749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46403 } } }
I20260806 18:39:11.823729 8576 sys_catalog.cc:458] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:11.824162 8586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:11.824093 8578 sys_catalog.cc:455] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b54c84ea3a604b4798522c413d7ca749. Latest consensus state: current_term: 1 leader_uuid: "b54c84ea3a604b4798522c413d7ca749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b54c84ea3a604b4798522c413d7ca749" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46403 } } }
I20260806 18:39:11.824801 8578 sys_catalog.cc:458] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:11.828979 8586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:11.836184 8586 catalog_manager.cc:1347] Generated new cluster ID: d7f37734b02d42f886cc1b6a9916d915
I20260806 18:39:11.836290 8586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:11.862387 8586 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:11.863890 8586 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:11.871982 8586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b54c84ea3a604b4798522c413d7ca749: Generated new TSK 0
I20260806 18:39:11.872797 8586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:11.883845 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 8491
W20260806 18:39:11.892424 8475 connection.cc:535] client connection to 127.0.0.1:46403 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46403): Connection reset by peer (error 104)
2026-08-06T18:39:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1534 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:11 dist-test-slave-8w9v krb5kdc[8598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:11 dist-test-slave-8w9v krb5kdc[8598](info): set up 2 sockets
Aug 06 18:39:11 dist-test-slave-8w9v krb5kdc[8598](info): commencing operation
krb5kdc: starting...
W20260806 18:39:12.967885 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:13 dist-test-slave-8w9v krb5kdc[8598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:13.107450 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34755
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:13.224611 8612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:13.224948 8612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:13.225025 8612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:13.229203 8612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:13.229326 8612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:13.229367 8612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:13.229408 8612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:13.229447 8612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:13.234997 8612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:13.236268 8612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:13.237357 8612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:13.247277 8618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:13.247292 8617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:13.247308 8620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:13.247638 8612 server_base.cc:1034] running on GCE node
I20260806 18:39:13.248407 8612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:13.249624 8612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:13.251022 8612 hybrid_clock.cc:648] HybridClock initialized: now 1786041553250992 us; error 68 us; skew 500 ppm
Aug 06 18:39:13 dist-test-slave-8w9v krb5kdc[8598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041553, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:13.254474 8612 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:13.255501 8612 webserver.cc:460] Webserver started at http://127.0.0.1:33973/ using document root <none> and password file <none>
I20260806 18:39:13.256242 8612 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:13.256349 8612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:13.256628 8612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:13.258884 8612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "2bde664c1b5e4de58b148d7a60193ca1"
format_stamp: "Formatted at 2026-08-06 18:39:13 on dist-test-slave-8w9v"
server_key: "966b983cde0d6bd0526996bd1128d6d9"
server_key_iv: "de87fa5612641706ab02c62eecffa5f0"
server_key_version: "encryptionkey@0"
I20260806 18:39:13.259505 8612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "2bde664c1b5e4de58b148d7a60193ca1"
format_stamp: "Formatted at 2026-08-06 18:39:13 on dist-test-slave-8w9v"
server_key: "966b983cde0d6bd0526996bd1128d6d9"
server_key_iv: "de87fa5612641706ab02c62eecffa5f0"
server_key_version: "encryptionkey@0"
I20260806 18:39:13.263636 8612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:39:13.266484 8629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:13.268425 8612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:13.268735 8612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "2bde664c1b5e4de58b148d7a60193ca1"
format_stamp: "Formatted at 2026-08-06 18:39:13 on dist-test-slave-8w9v"
server_key: "966b983cde0d6bd0526996bd1128d6d9"
server_key_iv: "de87fa5612641706ab02c62eecffa5f0"
server_key_version: "encryptionkey@0"
I20260806 18:39:13.268890 8612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:13.274009 8612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:13.277379 8612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:13.277613 8612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:13.285110 8612 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34755
I20260806 18:39:13.285143 8683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34755 every 8 connection(s)
I20260806 18:39:13.286101 8612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:13.289103 8684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:13.294734 8684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1: Bootstrap starting.
I20260806 18:39:13.295169 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 8612
I20260806 18:39:13.295307 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:13.295619 18934 external_mini_cluster.cc:1428] Setting key bc41b216f42741fa7843bc973b02fcf3
I20260806 18:39:13.297691 8684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:13.298475 8684 log.cc:824] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:13.302038 8684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1: No bootstrap required, opened a new log
I20260806 18:39:13.302743 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:13.297073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:59727 (local address 127.0.0.1:34755)
0806 18:39:13.297638 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:13.297667 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:13.297715 (+ 48us) server_negotiation.cc:407] Connection header received
0806 18:39:13.298472 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:13.298490 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:13.299174 (+ 684us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:13.300008 (+ 834us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:13.301276 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:13.301282 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:13.301294 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:13.301307 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:13.301367 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:13.301388 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:13.301505 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:13.301722 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 18:39:13.301957 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":138,"threads_started":1}
I20260806 18:39:13.306912 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:13.307580 8684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bde664c1b5e4de58b148d7a60193ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34755 } }
I20260806 18:39:13.307837 8684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:13.307912 8684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bde664c1b5e4de58b148d7a60193ca1, State: Initialized, Role: FOLLOWER
I20260806 18:39:13.308753 8684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bde664c1b5e4de58b148d7a60193ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34755 } }
I20260806 18:39:13.308897 8684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:13.308985 8684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:13.309098 8684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:13.310380 8684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bde664c1b5e4de58b148d7a60193ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34755 } }
I20260806 18:39:13.310806 8684 leader_election.cc:302] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bde664c1b5e4de58b148d7a60193ca1; no voters:
I20260806 18:39:13.311090 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:13.309370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:54375 (local address 127.0.0.1:34755)
0806 18:39:13.309515 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:13.309520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:13.309710 (+ 190us) server_negotiation.cc:407] Connection header received
0806 18:39:13.309817 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:13.309820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:13.309863 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:13.309951 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:13.310689 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:13.310693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:13.310696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:13.310708 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:13.310737 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:13.310750 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:13.310868 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:13.310912 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 18:39:13.310967 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:39:13.311177 8684 leader_election.cc:288] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:13.311434 8694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:13.311728 8694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [term 1 LEADER]: Becoming Leader. State: Replica: 2bde664c1b5e4de58b148d7a60193ca1, State: Running, Role: LEADER
I20260806 18:39:13.312155 8694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bde664c1b5e4de58b148d7a60193ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34755 } }
I20260806 18:39:13.312664 8684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:13.313920 8695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bde664c1b5e4de58b148d7a60193ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bde664c1b5e4de58b148d7a60193ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34755 } } }
I20260806 18:39:13.314059 8695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:13.314489 8704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:13.314450 8696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bde664c1b5e4de58b148d7a60193ca1. Latest consensus state: current_term: 1 leader_uuid: "2bde664c1b5e4de58b148d7a60193ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bde664c1b5e4de58b148d7a60193ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34755 } } }
I20260806 18:39:13.314597 8696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:13.317929 8704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:13.323405 8704 catalog_manager.cc:1347] Generated new cluster ID: 374bd825ab35492bb98d09bf8c0b3bae
I20260806 18:39:13.323482 8704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:13.350247 8704 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:13.350620 8704 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:13.361032 8704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bde664c1b5e4de58b148d7a60193ca1: Generated new TSK 0
I20260806 18:39:13.361780 8704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:13.366678 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 8612
2026-08-06T18:39:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:13 dist-test-slave-8w9v krb5kdc[8719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:13 dist-test-slave-8w9v krb5kdc[8719](info): set up 2 sockets
Aug 06 18:39:13 dist-test-slave-8w9v krb5kdc[8719](info): commencing operation
krb5kdc: starting...
W20260806 18:39:14.463521 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:14 dist-test-slave-8w9v krb5kdc[8719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:14.600476 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:14.718748 8732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:14.719099 8732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:14.719182 8732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:14.723368 8732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:14.723495 8732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:14.723536 8732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:14.723572 8732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:14.723613 8732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:14.729228 8732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:14.730453 8732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:14.731537 8732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:39:14.740845 8732 server_base.cc:1034] running on GCE node
W20260806 18:39:14.740898 8738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:14.740898 8737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:14.740931 8740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:14.741830 8732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:14.742936 8732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:14.744187 8732 hybrid_clock.cc:648] HybridClock initialized: now 1786041554744164 us; error 27 us; skew 500 ppm
Aug 06 18:39:14 dist-test-slave-8w9v krb5kdc[8719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041554, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:14.747013 8732 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:14.748031 8732 webserver.cc:460] Webserver started at http://127.0.0.1:42261/ using document root <none> and password file <none>
I20260806 18:39:14.748643 8732 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:14.748708 8732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:14.748879 8732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:14.750983 8732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "779d961873224d1db3fa53811ddcdf03"
format_stamp: "Formatted at 2026-08-06 18:39:14 on dist-test-slave-8w9v"
server_key: "7993ee5d30193307438f3fecaed58384"
server_key_iv: "e0ee1d0595182358b3bf244c609756b9"
server_key_version: "encryptionkey@0"
I20260806 18:39:14.751531 8732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "779d961873224d1db3fa53811ddcdf03"
format_stamp: "Formatted at 2026-08-06 18:39:14 on dist-test-slave-8w9v"
server_key: "7993ee5d30193307438f3fecaed58384"
server_key_iv: "e0ee1d0595182358b3bf244c609756b9"
server_key_version: "encryptionkey@0"
I20260806 18:39:14.755613 8732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:39:14.758800 8747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:14.760455 8732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:14.760601 8732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "779d961873224d1db3fa53811ddcdf03"
format_stamp: "Formatted at 2026-08-06 18:39:14 on dist-test-slave-8w9v"
server_key: "7993ee5d30193307438f3fecaed58384"
server_key_iv: "e0ee1d0595182358b3bf244c609756b9"
server_key_version: "encryptionkey@0"
I20260806 18:39:14.760721 8732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:14.781226 8732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:14.786509 8732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:14.786727 8732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:14.797178 8732 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37801
I20260806 18:39:14.797188 8803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37801 every 8 connection(s)
I20260806 18:39:14.798331 8732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:14.798873 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 8732
I20260806 18:39:14.799081 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:14.799579 18934 external_mini_cluster.cc:1428] Setting key 53b9c4771a33192d69a515c684ffa9ae
I20260806 18:39:14.802469 8804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:14.809477 8804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03: Bootstrap starting.
Aug 06 18:39:14 dist-test-slave-8w9v krb5kdc[8719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:39:14.812492 8804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:14.813473 8804 log.cc:824] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:14.815864 8804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03: No bootstrap required, opened a new log
I20260806 18:39:14.817754 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:14.801314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:54059 (local address 127.0.0.1:37801)
0806 18:39:14.801880 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:14.801898 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:14.801939 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:39:14.802815 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:14.802837 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:14.803653 (+ 816us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:14.804402 (+ 749us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:14.805289 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:14.806394 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:14.807333 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:14.807850 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:14.808068 (+ 218us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:14.811322 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:14.811329 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:14.811346 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:14.811395 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:14.813795 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:14.814486 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:14.814492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:14.814498 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:14.814647 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:14.815056 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:14.815064 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:14.815066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:14.815560 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:14.815811 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:14.816275 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 18:39:14.816680 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":174,"threads_started":1}
I20260806 18:39:14.820904 8804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d961873224d1db3fa53811ddcdf03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } }
I20260806 18:39:14.821187 8804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:14.821266 8804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779d961873224d1db3fa53811ddcdf03, State: Initialized, Role: FOLLOWER
I20260806 18:39:14.821415 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:14.821753 8804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d961873224d1db3fa53811ddcdf03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } }
I20260806 18:39:14.821885 8804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:14.821966 8804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:14.822212 8804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:14.823381 8804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d961873224d1db3fa53811ddcdf03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } }
I20260806 18:39:14.823764 8804 leader_election.cc:302] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 779d961873224d1db3fa53811ddcdf03; no voters:
I20260806 18:39:14.824148 8804 leader_election.cc:288] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:14.824267 8813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:14.824471 8813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [term 1 LEADER]: Becoming Leader. State: Replica: 779d961873224d1db3fa53811ddcdf03, State: Running, Role: LEADER
I20260806 18:39:14.824822 8813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d961873224d1db3fa53811ddcdf03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } }
I20260806 18:39:14.825706 8804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:14.825996 8813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "779d961873224d1db3fa53811ddcdf03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d961873224d1db3fa53811ddcdf03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } } }
I20260806 18:39:14.826114 8813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:14.826325 8813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 779d961873224d1db3fa53811ddcdf03. Latest consensus state: current_term: 1 leader_uuid: "779d961873224d1db3fa53811ddcdf03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779d961873224d1db3fa53811ddcdf03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } } }
I20260806 18:39:14.826412 8813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:14.826714 8818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:14.831292 8818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:14.837309 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:14.824242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:38613 (local address 127.0.0.1:37801)
0806 18:39:14.824372 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:14.824379 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:14.824689 (+ 310us) server_negotiation.cc:407] Connection header received
0806 18:39:14.824819 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:14.824822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:14.824876 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:14.824975 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:14.825969 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:14.826780 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:14.828101 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:14.828505 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:14.828598 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:14.831435 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:14.831440 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:14.831444 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:14.831488 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:14.834272 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:14.835468 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:14.835475 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:14.835477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:14.835609 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:14.836071 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:14.836075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:14.836077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:14.836371 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:14.836533 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:14.836997 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 18:39:14.837121 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:39:14.840183 8818 catalog_manager.cc:1347] Generated new cluster ID: 06dd47af8b47445d96a6ead67c0462c9
I20260806 18:39:14.840291 8818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:14.873642 8818 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:14.874856 8818 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:14.883469 8818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 779d961873224d1db3fa53811ddcdf03: Generated new TSK 0
I20260806 18:39:14.884718 8818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:14.898694 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 8732
W20260806 18:39:14.906268 8717 connection.cc:535] client connection to 127.0.0.1:37801 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37801): Connection reset by peer (error 104)
2026-08-06T18:39:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1530 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:14 dist-test-slave-8w9v krb5kdc[8836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:14 dist-test-slave-8w9v krb5kdc[8836](info): set up 2 sockets
Aug 06 18:39:14 dist-test-slave-8w9v krb5kdc[8836](info): commencing operation
krb5kdc: starting...
W20260806 18:39:15.996371 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:16 dist-test-slave-8w9v krb5kdc[8836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:16.133397 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43947
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:16.249173 8850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:16.249560 8850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:16.249637 8850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:16.253959 8850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:16.254078 8850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:16.254114 8850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:16.254153 8850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:16.254185 8850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:16.261493 8850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43947
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:16.262957 8850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:16.264221 8850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:16.274133 8855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:16.274114 8859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:16.274116 8857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:16.274557 8850 server_base.cc:1034] running on GCE node
I20260806 18:39:16.275213 8850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:16.276563 8850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:16.277760 8850 hybrid_clock.cc:648] HybridClock initialized: now 1786041556277729 us; error 47 us; skew 500 ppm
Aug 06 18:39:16 dist-test-slave-8w9v krb5kdc[8836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041556, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:16.285955 8850 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:16.287062 8850 webserver.cc:460] Webserver started at http://127.0.0.1:45955/ using document root <none> and password file <none>
I20260806 18:39:16.287691 8850 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:16.287789 8850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:16.288034 8850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:16.290325 8850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "34a65f15c2b14b32a51e29b0f1100bbc"
format_stamp: "Formatted at 2026-08-06 18:39:16 on dist-test-slave-8w9v"
server_key: "1545b30849debd47f78115c6b61542b2"
server_key_iv: "11fe62237d79b37fd56d3797e6ca5b45"
server_key_version: "encryptionkey@0"
I20260806 18:39:16.290946 8850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "34a65f15c2b14b32a51e29b0f1100bbc"
format_stamp: "Formatted at 2026-08-06 18:39:16 on dist-test-slave-8w9v"
server_key: "1545b30849debd47f78115c6b61542b2"
server_key_iv: "11fe62237d79b37fd56d3797e6ca5b45"
server_key_version: "encryptionkey@0"
I20260806 18:39:16.295142 8850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:39:16.298048 8866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:16.299378 8850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:39:16.299563 8850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "34a65f15c2b14b32a51e29b0f1100bbc"
format_stamp: "Formatted at 2026-08-06 18:39:16 on dist-test-slave-8w9v"
server_key: "1545b30849debd47f78115c6b61542b2"
server_key_iv: "11fe62237d79b37fd56d3797e6ca5b45"
server_key_version: "encryptionkey@0"
I20260806 18:39:16.299695 8850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:16.317255 8850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:16.321497 8850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:16.321751 8850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:16.330102 8850 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43947
I20260806 18:39:16.330133 8919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43947 every 8 connection(s)
I20260806 18:39:16.331161 8850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:16.334120 8920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:16.340022 8920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc: Bootstrap starting.
I20260806 18:39:16.341511 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 8850
I20260806 18:39:16.341655 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:16.341970 18934 external_mini_cluster.cc:1428] Setting key 3f6f992263f4976dddab3fec9c3f6898
I20260806 18:39:16.343091 8920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:16.343972 8920 log.cc:824] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:16.346089 8920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc: No bootstrap required, opened a new log
I20260806 18:39:16.349860 8920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a65f15c2b14b32a51e29b0f1100bbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260806 18:39:16.350126 8920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:16.350206 8920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a65f15c2b14b32a51e29b0f1100bbc, State: Initialized, Role: FOLLOWER
I20260806 18:39:16.350701 8920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a65f15c2b14b32a51e29b0f1100bbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260806 18:39:16.350859 8920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:16.350940 8920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:16.351068 8920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:16.351832 8923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:16.343211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:51119 (local address 127.0.0.1:43947)
0806 18:39:16.343770 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:16.343788 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:16.343827 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:39:16.344838 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:16.344868 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:16.345245 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:16.345606 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:16.346056 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:16.347079 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:16.348030 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:16.348605 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:16.348807 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:16.349831 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:16.349837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:16.349850 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:16.350357 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:16.350421 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:16.350448 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:16.350602 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:16.350814 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 18:39:16.350965 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":174,"threads_started":1}
I20260806 18:39:16.352185 8920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a65f15c2b14b32a51e29b0f1100bbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260806 18:39:16.352555 8920 leader_election.cc:302] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34a65f15c2b14b32a51e29b0f1100bbc; no voters:
I20260806 18:39:16.352918 8920 leader_election.cc:288] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:16.353111 8925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:16.353390 8925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [term 1 LEADER]: Becoming Leader. State: Replica: 34a65f15c2b14b32a51e29b0f1100bbc, State: Running, Role: LEADER
I20260806 18:39:16.353698 8925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a65f15c2b14b32a51e29b0f1100bbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } }
I20260806 18:39:16.354337 8920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:16.354967 8927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34a65f15c2b14b32a51e29b0f1100bbc. Latest consensus state: current_term: 1 leader_uuid: "34a65f15c2b14b32a51e29b0f1100bbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a65f15c2b14b32a51e29b0f1100bbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } } }
I20260806 18:39:16.355108 8927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:16.355180 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:16.355830 8926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34a65f15c2b14b32a51e29b0f1100bbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34a65f15c2b14b32a51e29b0f1100bbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43947 } } }
I20260806 18:39:16.355935 8926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:16.357409 8932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:16.360339 8932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:16.362537 8923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:16.357620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:33597 (local address 127.0.0.1:43947)
0806 18:39:16.357757 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:16.357761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:16.357915 (+ 154us) server_negotiation.cc:407] Connection header received
0806 18:39:16.358063 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:16.358067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:16.358117 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:16.358201 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:16.358476 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:16.359209 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:16.360435 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:16.360766 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:16.360881 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:16.361993 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:16.361997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:16.361999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:16.362013 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:16.362042 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:16.362056 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:16.362137 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:16.362360 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 18:39:16.362408 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:39:16.366446 8932 catalog_manager.cc:1347] Generated new cluster ID: fcde4e65e5a54bc4858a17ac685d3029
I20260806 18:39:16.366529 8932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:16.375016 8932 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:16.375984 8932 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:16.381726 8932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34a65f15c2b14b32a51e29b0f1100bbc: Generated new TSK 0
I20260806 18:39:16.382500 8932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:16.387490 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 8850
W20260806 18:39:16.394742 8834 connection.cc:535] client connection to 127.0.0.1:43947 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43947): Connection reset by peer (error 104)
2026-08-06T18:39:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:16 dist-test-slave-8w9v krb5kdc[8953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:16 dist-test-slave-8w9v krb5kdc[8953](info): set up 2 sockets
Aug 06 18:39:16 dist-test-slave-8w9v krb5kdc[8953](info): commencing operation
krb5kdc: starting...
W20260806 18:39:17.472210 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:17 dist-test-slave-8w9v krb5kdc[8953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:39:17.605662 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:17.721451 8968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:17.721820 8968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:17.721925 8968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:17.726538 8968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:17.726646 8968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:17.726686 8968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:17.726724 8968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:17.726756 8968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:17.732553 8968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38773
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:17.733749 8968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:17.734833 8968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:17.744196 8973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:17.744196 8974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:17.744196 8976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:17.744448 8968 server_base.cc:1034] running on GCE node
I20260806 18:39:17.745236 8968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:17.746670 8968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:17.752683 8968 hybrid_clock.cc:648] HybridClock initialized: now 1786041557752572 us; error 86 us; skew 500 ppm
Aug 06 18:39:17 dist-test-slave-8w9v krb5kdc[8953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041557, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:17.756455 8968 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:39:17.757519 8968 webserver.cc:460] Webserver started at http://127.0.0.1:41463/ using document root <none> and password file <none>
I20260806 18:39:17.758136 8968 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:17.758199 8968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:17.758455 8968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:17.760682 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "153c20a12b9146e7a965323d994a886a"
format_stamp: "Formatted at 2026-08-06 18:39:17 on dist-test-slave-8w9v"
server_key: "af882eca70c949fe8047dc046de3e773"
server_key_iv: "6fa51a233252af32380f881605840f37"
server_key_version: "encryptionkey@0"
I20260806 18:39:17.761230 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "153c20a12b9146e7a965323d994a886a"
format_stamp: "Formatted at 2026-08-06 18:39:17 on dist-test-slave-8w9v"
server_key: "af882eca70c949fe8047dc046de3e773"
server_key_iv: "6fa51a233252af32380f881605840f37"
server_key_version: "encryptionkey@0"
I20260806 18:39:17.765211 8968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:39:17.767746 8983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:17.769141 8968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:17.769296 8968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "153c20a12b9146e7a965323d994a886a"
format_stamp: "Formatted at 2026-08-06 18:39:17 on dist-test-slave-8w9v"
server_key: "af882eca70c949fe8047dc046de3e773"
server_key_iv: "6fa51a233252af32380f881605840f37"
server_key_version: "encryptionkey@0"
I20260806 18:39:17.769419 8968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:17.774335 8968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:17.778138 8968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:17.778378 8968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:17.786621 8968 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38773
I20260806 18:39:17.786644 9038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38773 every 8 connection(s)
I20260806 18:39:17.787729 8968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:17.792380 9039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:17.793496 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 8968
I20260806 18:39:17.793699 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:17.794059 18934 external_mini_cluster.cc:1428] Setting key 85a204e05ae363d4aa6df62e47c9cd59
I20260806 18:39:17.801558 9044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:17.795880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:36811 (local address 127.0.0.1:38773)
0806 18:39:17.796547 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:17.796566 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:17.796611 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:39:17.797412 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:17.797442 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:17.797806 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:17.798339 (+ 533us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:17.799829 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:17.799837 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:17.799855 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:17.799873 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:17.799965 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:17.799994 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:17.800241 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:17.800316 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 18:39:17.800570 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":235,"threads_started":1}
I20260806 18:39:17.803721 9039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a: Bootstrap starting.
I20260806 18:39:17.805778 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:17.806692 9039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:17.807632 9039 log.cc:824] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:17.810710 9044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:17.808473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:49737 (local address 127.0.0.1:38773)
0806 18:39:17.808622 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:17.808627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:17.808982 (+ 355us) server_negotiation.cc:407] Connection header received
0806 18:39:17.809206 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:17.809209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:17.809252 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:17.809374 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:17.810253 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:17.810256 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:17.810259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:17.810272 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:17.810306 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:17.810319 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:17.810414 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:17.810529 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 18:39:17.810578 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:39:17.811064 9039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a: No bootstrap required, opened a new log
I20260806 18:39:17.817255 9039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153c20a12b9146e7a965323d994a886a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260806 18:39:17.817653 9039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:17.817699 9039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153c20a12b9146e7a965323d994a886a, State: Initialized, Role: FOLLOWER
I20260806 18:39:17.818430 9039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153c20a12b9146e7a965323d994a886a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260806 18:39:17.818630 9039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:17.818732 9039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:17.818995 9039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:17.820942 9039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153c20a12b9146e7a965323d994a886a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260806 18:39:17.821444 9039 leader_election.cc:302] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 153c20a12b9146e7a965323d994a886a; no voters:
I20260806 18:39:17.821892 9039 leader_election.cc:288] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:17.822108 9052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:17.822463 9052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [term 1 LEADER]: Becoming Leader. State: Replica: 153c20a12b9146e7a965323d994a886a, State: Running, Role: LEADER
I20260806 18:39:17.822875 9052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153c20a12b9146e7a965323d994a886a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260806 18:39:17.823457 9039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:17.825166 9053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "153c20a12b9146e7a965323d994a886a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153c20a12b9146e7a965323d994a886a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } } }
I20260806 18:39:17.825227 9054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 153c20a12b9146e7a965323d994a886a. Latest consensus state: current_term: 1 leader_uuid: "153c20a12b9146e7a965323d994a886a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153c20a12b9146e7a965323d994a886a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } } }
I20260806 18:39:17.825330 9053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:17.825610 9054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:17.826660 9063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:17.830513 9063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:17.839839 9063 catalog_manager.cc:1347] Generated new cluster ID: 716618ace2ad4af180140355d4b40e90
I20260806 18:39:17.840065 9063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:17.850914 9063 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:17.851549 9063 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:17.859184 9063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 153c20a12b9146e7a965323d994a886a: Generated new TSK 0
I20260806 18:39:17.860073 9063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:17.866374 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 8968
2026-08-06T18:39:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:17 dist-test-slave-8w9v krb5kdc[9074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:17 dist-test-slave-8w9v krb5kdc[9074](info): set up 2 sockets
Aug 06 18:39:17 dist-test-slave-8w9v krb5kdc[9074](info): commencing operation
krb5kdc: starting...
W20260806 18:39:18.935307 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:19 dist-test-slave-8w9v krb5kdc[9074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:39:19.068435 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:35127
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:37567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:35127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:19.184507 9088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:19.184856 9088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:19.184939 9088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:19.189260 9088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:19.189365 9088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:19.189402 9088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:19.189445 9088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:19.189486 9088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:19.195137 9088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:37567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:35127
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:35127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:19.196432 9088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:19.197501 9088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:19.206468 9093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:19.206481 9096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:19.206470 9094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:19.206766 9088 server_base.cc:1034] running on GCE node
I20260806 18:39:19.207436 9088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:19.208660 9088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:19.209864 9088 hybrid_clock.cc:648] HybridClock initialized: now 1786041559209833 us; error 47 us; skew 500 ppm
Aug 06 18:39:19 dist-test-slave-8w9v krb5kdc[9074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041559, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:19.213198 9088 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:39:19.214183 9088 webserver.cc:460] Webserver started at http://127.18.125.190:38621/ using document root <none> and password file <none>
I20260806 18:39:19.214795 9088 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:19.214890 9088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:19.215130 9088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:19.217609 9088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "9ea8b654d6c945fca4a46de1ec341900"
format_stamp: "Formatted at 2026-08-06 18:39:19 on dist-test-slave-8w9v"
server_key: "fd04f569ad1732cc89e3abed5ec80f6e"
server_key_iv: "abe5bde18ea9d42cdf10b653b5e01003"
server_key_version: "encryptionkey@0"
I20260806 18:39:19.218252 9088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "9ea8b654d6c945fca4a46de1ec341900"
format_stamp: "Formatted at 2026-08-06 18:39:19 on dist-test-slave-8w9v"
server_key: "fd04f569ad1732cc89e3abed5ec80f6e"
server_key_iv: "abe5bde18ea9d42cdf10b653b5e01003"
server_key_version: "encryptionkey@0"
I20260806 18:39:19.222959 9088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:39:19.226131 9103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:19.227773 9088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:39:19.227952 9088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "9ea8b654d6c945fca4a46de1ec341900"
format_stamp: "Formatted at 2026-08-06 18:39:19 on dist-test-slave-8w9v"
server_key: "fd04f569ad1732cc89e3abed5ec80f6e"
server_key_iv: "abe5bde18ea9d42cdf10b653b5e01003"
server_key_version: "encryptionkey@0"
I20260806 18:39:19.228096 9088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:19.248783 9088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:19.252445 9088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:19.252679 9088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:19.260362 9088 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:35127
I20260806 18:39:19.260411 9159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:35127 every 8 connection(s)
I20260806 18:39:19.261399 9088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:19.264545 9160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:19.266518 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 9088
I20260806 18:39:19.266673 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:19.266963 18934 external_mini_cluster.cc:1428] Setting key d72edf43873d18e6a3c981c774e22544
I20260806 18:39:19.272224 9160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900: Bootstrap starting.
I20260806 18:39:19.275202 9160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:19.276057 9160 log.cc:824] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:39:19 dist-test-slave-8w9v krb5kdc[9074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:39:19.278759 9160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900: No bootstrap required, opened a new log
I20260806 18:39:19.283330 9160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea8b654d6c945fca4a46de1ec341900" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35127 } }
I20260806 18:39:19.283627 9160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:19.283720 9160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea8b654d6c945fca4a46de1ec341900, State: Initialized, Role: FOLLOWER
I20260806 18:39:19.284246 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:19.268590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37527 (local address 127.18.125.190:35127)
0806 18:39:19.269105 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:19.269125 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:19.269170 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:39:19.270110 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:19.270140 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:19.270666 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:19.271089 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:19.271802 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:19.273298 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:19.274256 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:19.274954 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:19.275168 (+ 214us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:19.278324 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:19.278331 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:19.278350 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:19.278387 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:19.280438 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:19.281233 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:19.281240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:19.281248 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:19.281376 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:19.281776 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:19.281779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:19.281780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:19.282260 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:19.282460 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:19.282874 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 18:39:19.283198 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":143,"threads_started":1}
I20260806 18:39:19.284253 9160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea8b654d6c945fca4a46de1ec341900" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35127 } }
I20260806 18:39:19.284571 9160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:19.284652 9160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:19.284756 9160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:19.285820 9160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea8b654d6c945fca4a46de1ec341900" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35127 } }
I20260806 18:39:19.286190 9160 leader_election.cc:302] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ea8b654d6c945fca4a46de1ec341900; no voters:
I20260806 18:39:19.286588 9160 leader_election.cc:288] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:19.286692 9165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:19.286958 9165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [term 1 LEADER]: Becoming Leader. State: Replica: 9ea8b654d6c945fca4a46de1ec341900, State: Running, Role: LEADER
I20260806 18:39:19.287391 9165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea8b654d6c945fca4a46de1ec341900" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35127 } }
I20260806 18:39:19.287621 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:19.288023 9160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:19.289165 9166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ea8b654d6c945fca4a46de1ec341900. Latest consensus state: current_term: 1 leader_uuid: "9ea8b654d6c945fca4a46de1ec341900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea8b654d6c945fca4a46de1ec341900" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35127 } } }
I20260806 18:39:19.289307 9166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:19.289647 9167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ea8b654d6c945fca4a46de1ec341900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea8b654d6c945fca4a46de1ec341900" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35127 } } }
I20260806 18:39:19.289743 9167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:19.290799 9179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:19.294180 9179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:19.300573 9179 catalog_manager.cc:1347] Generated new cluster ID: a8e2a8f8e064437c9151179a283dbc23
I20260806 18:39:19.300643 9179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:19.301923 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:19.290249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39779 (local address 127.18.125.190:35127)
0806 18:39:19.290407 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:19.290414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:19.290903 (+ 489us) server_negotiation.cc:407] Connection header received
0806 18:39:19.291110 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:19.291113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:19.291166 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:19.291257 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:19.291900 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:19.292753 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:19.293893 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:19.294216 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:19.294323 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:19.296285 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:19.296290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:19.296292 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:19.296320 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:19.298923 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:19.299499 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:19.299503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:19.299505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:19.299605 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:19.300158 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:19.300162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:19.300164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:19.300562 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:19.301237 (+ 675us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:19.301627 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 18:39:19.301744 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:39:19.323918 9179 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:19.325237 9179 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:19.333664 9179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ea8b654d6c945fca4a46de1ec341900: Generated new TSK 0
I20260806 18:39:19.334472 9179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:19.347638 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 9088
W20260806 18:39:19.354159 9072 connection.cc:535] client connection to 127.18.125.190:35127 recv error: Network error: failed to read from TLS socket (remote: 127.18.125.190:35127): Connection reset by peer (error 104)
2026-08-06T18:39:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:19 dist-test-slave-8w9v krb5kdc[9192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:19 dist-test-slave-8w9v krb5kdc[9192](info): set up 2 sockets
Aug 06 18:39:19 dist-test-slave-8w9v krb5kdc[9192](info): commencing operation
krb5kdc: starting...
W20260806 18:39:20.429840 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:20 dist-test-slave-8w9v krb5kdc[9192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:39:20.564404 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:36247
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:35679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:36247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:20.680317 9205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:20.680714 9205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:20.680819 9205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:20.685186 9205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:20.685322 9205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:20.685361 9205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:20.685398 9205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:20.685436 9205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:20.691674 9205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:35679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:36247
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:36247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:20.693048 9205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:20.694207 9205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:20.703848 9210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:20.703867 9211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:20.703850 9213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:20.704034 9205 server_base.cc:1034] running on GCE node
I20260806 18:39:20.705318 9205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:20.706549 9205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:20.707731 9205 hybrid_clock.cc:648] HybridClock initialized: now 1786041560707703 us; error 46 us; skew 500 ppm
Aug 06 18:39:20 dist-test-slave-8w9v krb5kdc[9192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041560, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:20.710685 9205 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:39:20.711654 9205 webserver.cc:460] Webserver started at http://127.18.125.190:36457/ using document root <none> and password file <none>
I20260806 18:39:20.712311 9205 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:20.712426 9205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:20.712657 9205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:20.714843 9205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "9e515ebf1182446d9fdbc43a2f5e0834"
format_stamp: "Formatted at 2026-08-06 18:39:20 on dist-test-slave-8w9v"
server_key: "d667f837981a7ff5af5e5c7e6ed8ee0b"
server_key_iv: "0dfd29d718703d4a430950207834c844"
server_key_version: "encryptionkey@0"
I20260806 18:39:20.715440 9205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "9e515ebf1182446d9fdbc43a2f5e0834"
format_stamp: "Formatted at 2026-08-06 18:39:20 on dist-test-slave-8w9v"
server_key: "d667f837981a7ff5af5e5c7e6ed8ee0b"
server_key_iv: "0dfd29d718703d4a430950207834c844"
server_key_version: "encryptionkey@0"
I20260806 18:39:20.719324 9205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:39:20.721938 9220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:20.723522 9205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:39:20.723678 9205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "9e515ebf1182446d9fdbc43a2f5e0834"
format_stamp: "Formatted at 2026-08-06 18:39:20 on dist-test-slave-8w9v"
server_key: "d667f837981a7ff5af5e5c7e6ed8ee0b"
server_key_iv: "0dfd29d718703d4a430950207834c844"
server_key_version: "encryptionkey@0"
I20260806 18:39:20.723799 9205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:20.740422 9205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:20.743989 9205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:20.744290 9205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:20.751889 9205 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:36247
I20260806 18:39:20.751951 9276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:36247 every 8 connection(s)
I20260806 18:39:20.753019 9205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:20.756215 9277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:20.761998 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 9205
I20260806 18:39:20.762169 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:20.762584 9277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834: Bootstrap starting.
I20260806 18:39:20.762881 18934 external_mini_cluster.cc:1428] Setting key fc4dd21db23055df8574765444f2c421
I20260806 18:39:20.766706 9277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:20.767689 9277 log.cc:824] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:20.770138 9277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834: No bootstrap required, opened a new log
I20260806 18:39:20.774492 9281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:20.765138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58201 (local address 127.18.125.190:36247)
0806 18:39:20.765638 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:20.765658 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:20.765704 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:39:20.766681 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:20.766706 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:20.767186 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:20.767593 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:20.768064 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:20.769485 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:20.770491 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:20.771175 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:20.771355 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:20.772239 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:20.772246 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:20.772263 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:20.772804 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:20.772875 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:20.772906 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:20.773040 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:20.773228 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:39:20.773388 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":157,"threads_started":1}
I20260806 18:39:20.775195 9277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e515ebf1182446d9fdbc43a2f5e0834" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36247 } }
I20260806 18:39:20.775418 9277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:20.775475 9277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e515ebf1182446d9fdbc43a2f5e0834, State: Initialized, Role: FOLLOWER
I20260806 18:39:20.775997 9277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e515ebf1182446d9fdbc43a2f5e0834" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36247 } }
I20260806 18:39:20.776185 9277 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:20.776257 9277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:20.776353 9277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:20.777587 9277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e515ebf1182446d9fdbc43a2f5e0834" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36247 } }
I20260806 18:39:20.777787 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:20.777976 9277 leader_election.cc:302] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e515ebf1182446d9fdbc43a2f5e0834; no voters:
I20260806 18:39:20.778298 9277 leader_election.cc:288] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:20.778470 9283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:20.778769 9283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [term 1 LEADER]: Becoming Leader. State: Replica: 9e515ebf1182446d9fdbc43a2f5e0834, State: Running, Role: LEADER
I20260806 18:39:20.779165 9283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e515ebf1182446d9fdbc43a2f5e0834" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36247 } }
I20260806 18:39:20.779698 9277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:20.780612 9283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e515ebf1182446d9fdbc43a2f5e0834. Latest consensus state: current_term: 1 leader_uuid: "9e515ebf1182446d9fdbc43a2f5e0834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e515ebf1182446d9fdbc43a2f5e0834" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36247 } } }
I20260806 18:39:20.780849 9283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:20.781141 9284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e515ebf1182446d9fdbc43a2f5e0834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e515ebf1182446d9fdbc43a2f5e0834" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36247 } } }
I20260806 18:39:20.781248 9284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834 [sys.catalog]: This master's current role is: LEADER
W20260806 18:39:20.785914 9303 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:20.786027 9303 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:20.786111 9296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:20.786551 9281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:20.780355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56685 (local address 127.18.125.190:36247)
0806 18:39:20.780509 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:20.780516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:20.781645 (+ 1129us) server_negotiation.cc:407] Connection header received
0806 18:39:20.781783 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:20.781785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:20.781830 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:20.781950 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:20.782443 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:20.783197 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:20.784353 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:20.784678 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:20.784755 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:20.786014 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:20.786018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:20.786022 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:20.786037 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:20.786071 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:20.786084 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:20.786186 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:20.786378 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 18:39:20.786420 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:39:20.787142 9296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:20.793311 9296 catalog_manager.cc:1347] Generated new cluster ID: 9fe7f32f57a149eaa54d3913428c50e8
I20260806 18:39:20.793413 9296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:20.810992 9296 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:20.812448 9296 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:20.821851 9296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e515ebf1182446d9fdbc43a2f5e0834: Generated new TSK 0
I20260806 18:39:20.823240 9296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:20.833951 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 9205
W20260806 18:39:20.842348 9190 connection.cc:535] client connection to 127.18.125.190:36247 recv error: Network error: failed to read from TLS socket (remote: 127.18.125.190:36247): Connection reset by peer (error 104)
2026-08-06T18:39:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:20 dist-test-slave-8w9v krb5kdc[9310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:20 dist-test-slave-8w9v krb5kdc[9310](info): set up 2 sockets
Aug 06 18:39:20 dist-test-slave-8w9v krb5kdc[9310](info): commencing operation
krb5kdc: starting...
W20260806 18:39:21.930518 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:22 dist-test-slave-8w9v krb5kdc[9310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:39:22.067488 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:36451
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:36451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:22.186761 9323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:22.187115 9323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:22.187192 9323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:22.191447 9323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:22.191550 9323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:22.191588 9323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:22.191646 9323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:22.191685 9323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:22.197304 9323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:36451
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:36451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:22.198465 9323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:22.199536 9323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:22.209051 9331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:22.209051 9329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:22.209051 9328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:22.209468 9323 server_base.cc:1034] running on GCE node
I20260806 18:39:22.210260 9323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:22.211648 9323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:22.212864 9323 hybrid_clock.cc:648] HybridClock initialized: now 1786041562212832 us; error 49 us; skew 500 ppm
Aug 06 18:39:22 dist-test-slave-8w9v krb5kdc[9310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041562, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:22.221290 9323 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:39:22.222380 9323 webserver.cc:460] Webserver started at http://127.18.125.190:42183/ using document root <none> and password file <none>
I20260806 18:39:22.223007 9323 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:22.223102 9323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:22.223353 9323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:22.225595 9323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "14536fc631814c4fa6db553a4140e52f"
format_stamp: "Formatted at 2026-08-06 18:39:22 on dist-test-slave-8w9v"
server_key: "2007106321b72792e1e8567368c1778c"
server_key_iv: "17d102532db3237204dc56efb3d25feb"
server_key_version: "encryptionkey@0"
I20260806 18:39:22.226205 9323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "14536fc631814c4fa6db553a4140e52f"
format_stamp: "Formatted at 2026-08-06 18:39:22 on dist-test-slave-8w9v"
server_key: "2007106321b72792e1e8567368c1778c"
server_key_iv: "17d102532db3237204dc56efb3d25feb"
server_key_version: "encryptionkey@0"
I20260806 18:39:22.230556 9323 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 18:39:22.233409 9338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:22.234879 9323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:22.235074 9323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "14536fc631814c4fa6db553a4140e52f"
format_stamp: "Formatted at 2026-08-06 18:39:22 on dist-test-slave-8w9v"
server_key: "2007106321b72792e1e8567368c1778c"
server_key_iv: "17d102532db3237204dc56efb3d25feb"
server_key_version: "encryptionkey@0"
I20260806 18:39:22.235203 9323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:22.240283 9323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:22.243975 9323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:22.244248 9323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:22.252959 9323 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:36451
I20260806 18:39:22.252980 9394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:36451 every 8 connection(s)
I20260806 18:39:22.254122 9323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:22.254998 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 9323
I20260806 18:39:22.255163 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:22.255512 18934 external_mini_cluster.cc:1428] Setting key 0a2d3a490b9d0db8cbc27c5942eb5da6
I20260806 18:39:22.258143 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:22.264339 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:22.257294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46585 (local address 127.18.125.190:36451)
0806 18:39:22.258193 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:22.258247 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:22.258339 (+ 92us) server_negotiation.cc:407] Connection header received
0806 18:39:22.259594 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:22.259622 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:22.260110 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:22.260801 (+ 691us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:22.262587 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:22.262593 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:22.262610 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:22.262624 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:22.262689 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:22.262711 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:22.262921 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:22.263092 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 18:39:22.263295 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":163,"threads_started":1}
I20260806 18:39:22.266798 9395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f: Bootstrap starting.
I20260806 18:39:22.269485 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:22.270624 9395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:22.271512 9395 log.cc:824] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:22.273615 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:22.271865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60159 (local address 127.18.125.190:36451)
0806 18:39:22.272045 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:22.272050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:22.272218 (+ 168us) server_negotiation.cc:407] Connection header received
0806 18:39:22.272341 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:22.272344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:22.272383 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:22.272492 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:22.273223 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:22.273226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:22.273228 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:22.273239 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:22.273265 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:22.273275 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:22.273347 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:22.273443 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 18:39:22.273476 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:39:22.274376 9395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f: No bootstrap required, opened a new log
I20260806 18:39:22.278362 9395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14536fc631814c4fa6db553a4140e52f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36451 } }
I20260806 18:39:22.278754 9395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:22.278821 9395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14536fc631814c4fa6db553a4140e52f, State: Initialized, Role: FOLLOWER
I20260806 18:39:22.279304 9395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14536fc631814c4fa6db553a4140e52f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36451 } }
I20260806 18:39:22.279474 9395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:22.279565 9395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:22.279675 9395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:22.280833 9395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14536fc631814c4fa6db553a4140e52f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36451 } }
I20260806 18:39:22.281211 9395 leader_election.cc:302] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14536fc631814c4fa6db553a4140e52f; no voters:
I20260806 18:39:22.281581 9395 leader_election.cc:288] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:22.281739 9405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:22.281983 9405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [term 1 LEADER]: Becoming Leader. State: Replica: 14536fc631814c4fa6db553a4140e52f, State: Running, Role: LEADER
I20260806 18:39:22.282378 9405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14536fc631814c4fa6db553a4140e52f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36451 } }
I20260806 18:39:22.282971 9395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:22.284651 9406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14536fc631814c4fa6db553a4140e52f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14536fc631814c4fa6db553a4140e52f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36451 } } }
I20260806 18:39:22.284850 9406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:22.285192 9407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14536fc631814c4fa6db553a4140e52f. Latest consensus state: current_term: 1 leader_uuid: "14536fc631814c4fa6db553a4140e52f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14536fc631814c4fa6db553a4140e52f" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 36451 } } }
I20260806 18:39:22.285364 9407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:22.285670 9415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:22.289057 9415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:22.295142 9415 catalog_manager.cc:1347] Generated new cluster ID: 6af646d5cd0f422398be56aa6cb33837
I20260806 18:39:22.295233 9415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:22.307441 9415 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:22.307866 9415 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:22.313382 9415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14536fc631814c4fa6db553a4140e52f: Generated new TSK 0
I20260806 18:39:22.314354 9415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:22.324095 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 9323
2026-08-06T18:39:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:22 dist-test-slave-8w9v krb5kdc[9428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:22 dist-test-slave-8w9v krb5kdc[9428](info): set up 2 sockets
Aug 06 18:39:22 dist-test-slave-8w9v krb5kdc[9428](info): commencing operation
krb5kdc: starting...
W20260806 18:39:23.403049 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:23 dist-test-slave-8w9v krb5kdc[9428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:39:23.560974 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34699
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:34699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:23.677713 9444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:23.678030 9444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:23.678076 9444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:23.682456 9444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:23.682549 9444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:23.682570 9444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:23.682590 9444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:23.682605 9444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:23.688534 9444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:34699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:34699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:23.689759 9444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:23.690886 9444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:23.700109 9452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:23.700103 9450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:23.700232 9444 server_base.cc:1034] running on GCE node
W20260806 18:39:23.700095 9449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:23.700982 9444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:23.702164 9444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:23.703339 9444 hybrid_clock.cc:648] HybridClock initialized: now 1786041563703308 us; error 48 us; skew 500 ppm
Aug 06 18:39:23 dist-test-slave-8w9v krb5kdc[9428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041563, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:23.706415 9444 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:39:23.707623 9444 webserver.cc:460] Webserver started at http://127.18.125.190:37981/ using document root <none> and password file <none>
I20260806 18:39:23.708328 9444 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:23.708428 9444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:23.708664 9444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:23.710835 9444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "7590270639db4acaa6116704df066a50"
format_stamp: "Formatted at 2026-08-06 18:39:23 on dist-test-slave-8w9v"
server_key: "1ac539bf445e84ca1a488302a5883b40"
server_key_iv: "4809773cb3052ef34bc25b9c4501858e"
server_key_version: "encryptionkey@0"
I20260806 18:39:23.711441 9444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "7590270639db4acaa6116704df066a50"
format_stamp: "Formatted at 2026-08-06 18:39:23 on dist-test-slave-8w9v"
server_key: "1ac539bf445e84ca1a488302a5883b40"
server_key_iv: "4809773cb3052ef34bc25b9c4501858e"
server_key_version: "encryptionkey@0"
I20260806 18:39:23.715520 9444 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 18:39:23.718329 9460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:23.719630 9444 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:39:23.719786 9444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "7590270639db4acaa6116704df066a50"
format_stamp: "Formatted at 2026-08-06 18:39:23 on dist-test-slave-8w9v"
server_key: "1ac539bf445e84ca1a488302a5883b40"
server_key_iv: "4809773cb3052ef34bc25b9c4501858e"
server_key_version: "encryptionkey@0"
I20260806 18:39:23.719930 9444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:23.735615 9444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:23.739163 9444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:23.739387 9444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:23.747416 9444 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:34699
I20260806 18:39:23.747465 9513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:34699 every 8 connection(s)
I20260806 18:39:23.748703 9444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:23.752386 9514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:23.758316 9514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50: Bootstrap starting.
I20260806 18:39:23.758962 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 9444
I20260806 18:39:23.759121 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:23.759392 18934 external_mini_cluster.cc:1428] Setting key 30ef13956e74aee03062a9288fa2116a
I20260806 18:39:23.761395 9514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:23.762223 9514 log.cc:824] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:23.764662 9514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50: No bootstrap required, opened a new log
Aug 06 18:39:23 dist-test-slave-8w9v krb5kdc[9428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:39:23.769753 9514 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7590270639db4acaa6116704df066a50" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34699 } }
I20260806 18:39:23.770040 9514 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:23.770107 9514 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7590270639db4acaa6116704df066a50, State: Initialized, Role: FOLLOWER
I20260806 18:39:23.770617 9514 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7590270639db4acaa6116704df066a50" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34699 } }
I20260806 18:39:23.770746 9514 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:23.770831 9514 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:23.770964 9514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:23.771979 9514 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7590270639db4acaa6116704df066a50" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34699 } }
I20260806 18:39:23.772387 9514 leader_election.cc:302] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7590270639db4acaa6116704df066a50; no voters:
I20260806 18:39:23.772774 9514 leader_election.cc:288] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:23.772861 9519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:23.773087 9519 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [term 1 LEADER]: Becoming Leader. State: Replica: 7590270639db4acaa6116704df066a50, State: Running, Role: LEADER
I20260806 18:39:23.773420 9519 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7590270639db4acaa6116704df066a50" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34699 } }
I20260806 18:39:23.774269 9514 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:23.775357 9521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7590270639db4acaa6116704df066a50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7590270639db4acaa6116704df066a50" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34699 } } }
I20260806 18:39:23.775485 9521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:23.775419 9520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7590270639db4acaa6116704df066a50. Latest consensus state: current_term: 1 leader_uuid: "7590270639db4acaa6116704df066a50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7590270639db4acaa6116704df066a50" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 34699 } } }
I20260806 18:39:23.775550 9520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:23.775992 9528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:23.776206 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:23.760843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34155 (local address 127.18.125.190:34699)
0806 18:39:23.761308 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:23.761325 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:23.761362 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:39:23.762120 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:23.762139 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:23.762555 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:23.762971 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:23.763873 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:23.765010 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:23.766179 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:23.766680 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:23.766807 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:23.769918 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:23.769924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:23.769940 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:23.769975 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:23.771940 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:23.772479 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:23.772486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:23.772492 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:23.772621 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:23.773013 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:23.773016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:23.773018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:23.773730 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:23.773941 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:23.774260 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 18:39:23.774841 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":152,"threads_started":1}
I20260806 18:39:23.779409 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:23.779811 9528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:23.786613 9528 catalog_manager.cc:1347] Generated new cluster ID: d14c2f850cc64254b2c56a0f71f74720
I20260806 18:39:23.786702 9528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:23.793910 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:23.784868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52435 (local address 127.18.125.190:34699)
0806 18:39:23.784977 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:23.784982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:23.785402 (+ 420us) server_negotiation.cc:407] Connection header received
0806 18:39:23.785538 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:23.785542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:23.785599 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:23.785697 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:23.786650 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:23.787245 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:23.788279 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:23.788568 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:23.788696 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:23.790288 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:23.790292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:23.790295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:23.790335 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:23.791998 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:23.792532 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:23.792536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:23.792537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:23.792631 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:23.792936 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:23.792940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:23.792942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:23.793158 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:23.793305 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:23.793588 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 18:39:23.793719 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 18:39:23.804673 9528 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:23.806082 9528 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:23.816711 9528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7590270639db4acaa6116704df066a50: Generated new TSK 0
I20260806 18:39:23.817567 9528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:23.830577 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 9444
2026-08-06T18:39:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:23 dist-test-slave-8w9v krb5kdc[9546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:23 dist-test-slave-8w9v krb5kdc[9546](info): set up 2 sockets
Aug 06 18:39:23 dist-test-slave-8w9v krb5kdc[9546](info): commencing operation
krb5kdc: starting...
W20260806 18:39:24.949141 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:25 dist-test-slave-8w9v krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:39:25.097196 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:35949
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:42465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:35949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:25.217440 9561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:25.217809 9561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:25.217901 9561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:25.222172 9561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:25.222303 9561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:25.222350 9561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:25.222391 9561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:25.222427 9561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:25.228189 9561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:42465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:35949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:35949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:25.229449 9561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:25.230585 9561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:25.241667 9567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:25.241914 9561 server_base.cc:1034] running on GCE node
W20260806 18:39:25.241667 9566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:25.241803 9570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:25.243023 9561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:25.244629 9561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:25.245846 9561 hybrid_clock.cc:648] HybridClock initialized: now 1786041565245805 us; error 63 us; skew 500 ppm
Aug 06 18:39:25 dist-test-slave-8w9v krb5kdc[9546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041565, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:25.249718 9561 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:39:25.250815 9561 webserver.cc:460] Webserver started at http://127.18.125.190:42013/ using document root <none> and password file <none>
I20260806 18:39:25.251411 9561 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:25.251506 9561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:25.251696 9561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:25.254004 9561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "f37f2118271d4ece955d9b99ce01c9f4"
format_stamp: "Formatted at 2026-08-06 18:39:25 on dist-test-slave-8w9v"
server_key: "f9ee5a14cafdd81397c0b4cb75c6305f"
server_key_iv: "dbdf37e55dbb0d9c8715376a89000b41"
server_key_version: "encryptionkey@0"
I20260806 18:39:25.254652 9561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "f37f2118271d4ece955d9b99ce01c9f4"
format_stamp: "Formatted at 2026-08-06 18:39:25 on dist-test-slave-8w9v"
server_key: "f9ee5a14cafdd81397c0b4cb75c6305f"
server_key_iv: "dbdf37e55dbb0d9c8715376a89000b41"
server_key_version: "encryptionkey@0"
I20260806 18:39:25.258921 9561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 18:39:25.261982 9579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:25.263494 9561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:39:25.263674 9561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "f37f2118271d4ece955d9b99ce01c9f4"
format_stamp: "Formatted at 2026-08-06 18:39:25 on dist-test-slave-8w9v"
server_key: "f9ee5a14cafdd81397c0b4cb75c6305f"
server_key_iv: "dbdf37e55dbb0d9c8715376a89000b41"
server_key_version: "encryptionkey@0"
I20260806 18:39:25.263815 9561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:25.277945 9561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:25.281920 9561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:25.282186 9561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:25.291116 9561 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:35949
I20260806 18:39:25.291164 9632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:35949 every 8 connection(s)
I20260806 18:39:25.292238 9561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:25.295396 9633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:25.295434 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 9561
I20260806 18:39:25.295568 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:25.295914 18934 external_mini_cluster.cc:1428] Setting key d3c4703ee0d7f239bdea9ee15fec1a75
I20260806 18:39:25.302949 9633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4: Bootstrap starting.
I20260806 18:39:25.305675 9633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:25.305754 9636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:25.297530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32873 (local address 127.18.125.190:35949)
0806 18:39:25.298027 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:25.298054 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:25.298092 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:39:25.298824 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:25.298844 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:25.299212 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:25.299509 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:25.299887 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:25.300992 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:25.302273 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:25.302776 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:25.302902 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:25.303841 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:25.303846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:25.303860 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:25.304358 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:25.304421 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:25.304446 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:25.304527 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:25.304704 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 18:39:25.304987 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":169,"threads_started":1}
I20260806 18:39:25.306654 9633 log.cc:824] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:25.308902 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:25.309115 9633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4: No bootstrap required, opened a new log
I20260806 18:39:25.313133 9633 raft_consensus.cc:348] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37f2118271d4ece955d9b99ce01c9f4" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35949 } }
I20260806 18:39:25.313565 9633 raft_consensus.cc:374] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:25.313653 9633 raft_consensus.cc:729] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f37f2118271d4ece955d9b99ce01c9f4, State: Initialized, Role: FOLLOWER
I20260806 18:39:25.314209 9633 consensus_queue.cc:260] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37f2118271d4ece955d9b99ce01c9f4" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35949 } }
I20260806 18:39:25.314410 9633 raft_consensus.cc:388] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:25.314466 9633 raft_consensus.cc:482] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:25.314540 9633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:25.317065 9633 raft_consensus.cc:504] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37f2118271d4ece955d9b99ce01c9f4" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35949 } }
I20260806 18:39:25.317607 9633 leader_election.cc:302] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f37f2118271d4ece955d9b99ce01c9f4; no voters:
I20260806 18:39:25.318081 9633 leader_election.cc:288] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:25.318385 9643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:25.318894 9643 raft_consensus.cc:686] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [term 1 LEADER]: Becoming Leader. State: Replica: f37f2118271d4ece955d9b99ce01c9f4, State: Running, Role: LEADER
I20260806 18:39:25.319420 9643 consensus_queue.cc:237] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37f2118271d4ece955d9b99ce01c9f4" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35949 } }
I20260806 18:39:25.320199 9633 sys_catalog.cc:564] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:25.321909 9644 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f37f2118271d4ece955d9b99ce01c9f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37f2118271d4ece955d9b99ce01c9f4" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35949 } } }
I20260806 18:39:25.322099 9644 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:25.322279 9636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:25.311064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49649 (local address 127.18.125.190:35949)
0806 18:39:25.311211 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:25.311218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:25.313945 (+ 2727us) server_negotiation.cc:407] Connection header received
0806 18:39:25.314724 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:25.314765 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:25.314915 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:25.315243 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:25.316302 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:25.317732 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:25.319444 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:25.319815 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:25.319981 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:25.321565 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:25.321573 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:25.321578 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:25.321604 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:25.321650 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:25.321673 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:25.321741 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:25.321996 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 18:39:25.322103 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:39:25.322522 9645 sys_catalog.cc:455] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f37f2118271d4ece955d9b99ce01c9f4. Latest consensus state: current_term: 1 leader_uuid: "f37f2118271d4ece955d9b99ce01c9f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f37f2118271d4ece955d9b99ce01c9f4" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 35949 } } }
I20260806 18:39:25.322618 9645 sys_catalog.cc:458] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4 [sys.catalog]: This master's current role is: LEADER
W20260806 18:39:25.326572 9659 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:25.326690 9659 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:25.326768 9652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:25.327505 9652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:25.333901 9652 catalog_manager.cc:1347] Generated new cluster ID: ce0a415847c84b53aedb35c5ffd7e1b6
I20260806 18:39:25.333999 9652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:25.354286 9652 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:25.355660 9652 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:25.362560 9652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f37f2118271d4ece955d9b99ce01c9f4: Generated new TSK 0
I20260806 18:39:25.364004 9652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:25.372359 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 9561
2026-08-06T18:39:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1539 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:25 dist-test-slave-8w9v krb5kdc[9665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:25 dist-test-slave-8w9v krb5kdc[9665](info): set up 2 sockets
Aug 06 18:39:25 dist-test-slave-8w9v krb5kdc[9665](info): commencing operation
krb5kdc: starting...
W20260806 18:39:26.473990 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:26 dist-test-slave-8w9v krb5kdc[9665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:39:26.621531 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:43889
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:45963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:43889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:26.746168 9678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:26.746519 9678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:26.746568 9678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:26.751279 9678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:26.751390 9678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:26.751416 9678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:26.751439 9678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:26.751459 9678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:26.757813 9678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:45963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:43889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:43889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:26.759037 9678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:26.760236 9678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:26.769785 9684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:26.769853 9683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:26.769987 9678 server_base.cc:1034] running on GCE node
W20260806 18:39:26.769791 9686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:26.770763 9678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:26.771908 9678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:26.773074 9678 hybrid_clock.cc:648] HybridClock initialized: now 1786041566773057 us; error 40 us; skew 500 ppm
Aug 06 18:39:26 dist-test-slave-8w9v krb5kdc[9665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041566, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:26.776209 9678 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:39:26.777153 9678 webserver.cc:460] Webserver started at http://127.18.125.190:43953/ using document root <none> and password file <none>
I20260806 18:39:26.777760 9678 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:26.777854 9678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:26.778103 9678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:26.780422 9678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "6c6b0f73be744002a7064e0b9a1b9f26"
format_stamp: "Formatted at 2026-08-06 18:39:26 on dist-test-slave-8w9v"
server_key: "e8c60533df0b7c40e392ca91cc719102"
server_key_iv: "c2faf838473d147bd270faeaa7ad5681"
server_key_version: "encryptionkey@0"
I20260806 18:39:26.781029 9678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "6c6b0f73be744002a7064e0b9a1b9f26"
format_stamp: "Formatted at 2026-08-06 18:39:26 on dist-test-slave-8w9v"
server_key: "e8c60533df0b7c40e392ca91cc719102"
server_key_iv: "c2faf838473d147bd270faeaa7ad5681"
server_key_version: "encryptionkey@0"
I20260806 18:39:26.785094 9678 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:39:26.787994 9694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:26.789515 9678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:26.789697 9678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "6c6b0f73be744002a7064e0b9a1b9f26"
format_stamp: "Formatted at 2026-08-06 18:39:26 on dist-test-slave-8w9v"
server_key: "e8c60533df0b7c40e392ca91cc719102"
server_key_iv: "c2faf838473d147bd270faeaa7ad5681"
server_key_version: "encryptionkey@0"
I20260806 18:39:26.789867 9678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:26.794963 9678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:26.798342 9678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:26.798568 9678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:26.807399 9678 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:43889
I20260806 18:39:26.807425 9752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:43889 every 8 connection(s)
I20260806 18:39:26.808576 9678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:26.809429 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 9678
I20260806 18:39:26.809581 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:26.809872 18934 external_mini_cluster.cc:1428] Setting key c2ec2f19f521566ac9b8e0bbe65bbb28
I20260806 18:39:26.812310 9753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:26.816358 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:26.811444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36037 (local address 127.18.125.190:43889)
0806 18:39:26.811945 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:26.811967 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:26.812016 (+ 49us) server_negotiation.cc:407] Connection header received
0806 18:39:26.813027 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:26.813052 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:26.813455 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:26.813868 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:26.814685 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:26.814692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:26.814708 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:26.814721 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:26.814791 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:26.814823 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:26.814984 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:26.815059 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 18:39:26.815223 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":132,"threads_started":1}
I20260806 18:39:26.819367 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:26.820147 9753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26: Bootstrap starting.
I20260806 18:39:26.823400 9753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:26.824184 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:26.822070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56081 (local address 127.18.125.190:43889)
0806 18:39:26.822218 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:26.822223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:26.822443 (+ 220us) server_negotiation.cc:407] Connection header received
0806 18:39:26.822575 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:26.822577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:26.822613 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:26.822683 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:26.823664 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:26.823669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:26.823672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:26.823687 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:26.823725 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:26.823742 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:26.823839 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:26.823973 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 18:39:26.824021 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:39:26.824558 9753 log.cc:824] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:26.826812 9753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26: No bootstrap required, opened a new log
I20260806 18:39:26.831380 9753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6b0f73be744002a7064e0b9a1b9f26" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 43889 } }
I20260806 18:39:26.834014 9753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:26.834124 9753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6b0f73be744002a7064e0b9a1b9f26, State: Initialized, Role: FOLLOWER
I20260806 18:39:26.835686 9753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6b0f73be744002a7064e0b9a1b9f26" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 43889 } }
I20260806 18:39:26.835914 9753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:26.836045 9753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:26.836241 9753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:26.837466 9753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6b0f73be744002a7064e0b9a1b9f26" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 43889 } }
I20260806 18:39:26.837913 9753 leader_election.cc:302] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c6b0f73be744002a7064e0b9a1b9f26; no voters:
I20260806 18:39:26.838308 9753 leader_election.cc:288] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:26.838500 9764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:26.838842 9764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [term 1 LEADER]: Becoming Leader. State: Replica: 6c6b0f73be744002a7064e0b9a1b9f26, State: Running, Role: LEADER
I20260806 18:39:26.839244 9764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6b0f73be744002a7064e0b9a1b9f26" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 43889 } }
I20260806 18:39:26.839895 9753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:26.842381 9765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c6b0f73be744002a7064e0b9a1b9f26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6b0f73be744002a7064e0b9a1b9f26" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 43889 } } }
I20260806 18:39:26.842548 9765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:26.842777 9766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c6b0f73be744002a7064e0b9a1b9f26. Latest consensus state: current_term: 1 leader_uuid: "6c6b0f73be744002a7064e0b9a1b9f26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c6b0f73be744002a7064e0b9a1b9f26" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 43889 } } }
I20260806 18:39:26.843452 9766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26 [sys.catalog]: This master's current role is: LEADER
W20260806 18:39:26.846321 9778 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:26.846415 9778 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:26.846522 9779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:26.847462 9779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:26.854125 9779 catalog_manager.cc:1347] Generated new cluster ID: 898f9f73b10b4744aaa8d5ce3f258ad3
I20260806 18:39:26.854238 9779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:26.872821 9779 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:26.873320 9779 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:26.880479 9779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c6b0f73be744002a7064e0b9a1b9f26: Generated new TSK 0
I20260806 18:39:26.881423 9779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:26.898612 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 9678
2026-08-06T18:39:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1526 ms)
[----------] 18 tests from AuthTokenIssuingTest (27121 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:26 dist-test-slave-8w9v krb5kdc[9785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:26 dist-test-slave-8w9v krb5kdc[9785](info): set up 2 sockets
Aug 06 18:39:26 dist-test-slave-8w9v krb5kdc[9785](info): commencing operation
krb5kdc: starting...
W20260806 18:39:28.005110 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:39:28.144202 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:42025
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:42025,127.18.125.189:45035,127.18.125.188:37663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:28.264727 9799 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:39:28.265094 9799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:28.265149 9799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:28.265230 9799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:28.269639 9799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:28.269730 9799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:28.269752 9799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:28.269770 9799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:28.269788 9799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:28.276767 9799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:42025,127.18.125.189:45035,127.18.125.188:37663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:42025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:28.278057 9799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:28.279192 9799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:28.288453 9804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:28.288466 9805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:28.288511 9799 server_base.cc:1034] running on GCE node
W20260806 18:39:28.288479 9807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:28.289374 9799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:28.290603 9799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:28.291800 9799 hybrid_clock.cc:648] HybridClock initialized: now 1786041568291769 us; error 46 us; skew 500 ppm
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:28.300523 9799 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:39:28.301723 9799 webserver.cc:460] Webserver started at http://127.18.125.190:33351/ using document root <none> and password file <none>
I20260806 18:39:28.302395 9799 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:28.302471 9799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:28.302707 9799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:28.305158 9799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "7c8ae76b9fac444ca80983ebd6f594d7"
format_stamp: "Formatted at 2026-08-06 18:39:28 on dist-test-slave-8w9v"
server_key: "fdc3039af1e447a3f02a0097218ce37d"
server_key_iv: "5f874e5470eb8164000fbd44a60c9213"
server_key_version: "encryptionkey@0"
I20260806 18:39:28.305781 9799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "7c8ae76b9fac444ca80983ebd6f594d7"
format_stamp: "Formatted at 2026-08-06 18:39:28 on dist-test-slave-8w9v"
server_key: "fdc3039af1e447a3f02a0097218ce37d"
server_key_iv: "5f874e5470eb8164000fbd44a60c9213"
server_key_version: "encryptionkey@0"
I20260806 18:39:28.310117 9799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:39:28.313023 9815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:28.314399 9799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:39:28.314517 9799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "7c8ae76b9fac444ca80983ebd6f594d7"
format_stamp: "Formatted at 2026-08-06 18:39:28 on dist-test-slave-8w9v"
server_key: "fdc3039af1e447a3f02a0097218ce37d"
server_key_iv: "5f874e5470eb8164000fbd44a60c9213"
server_key_version: "encryptionkey@0"
I20260806 18:39:28.314618 9799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:28.326555 9799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:28.330009 9799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:28.330183 9799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:28.337778 9799 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:42025
I20260806 18:39:28.338128 9871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:42025 every 8 connection(s)
I20260806 18:39:28.338871 9799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:28.342136 9872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:28.342319 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 9799
I20260806 18:39:28.342478 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:28.342794 18934 external_mini_cluster.cc:1428] Setting key d7e929b0dbce6d89da002abd0ba6c957
I20260806 18:39:28.346717 9872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.125.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.189@KRBTEST.COM" created.
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:39:28.364743 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.349962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53370 (local address 127.18.125.190:42025)
0806 18:39:28.350645 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.350666 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.351060 (+ 394us) server_negotiation.cc:407] Connection header received
0806 18:39:28.352371 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.352375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.352586 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.352793 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.353682 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.354613 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.355674 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.356103 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.356308 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.359337 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.359344 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.359361 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.359384 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.361209 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.361719 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.361732 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.361733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.361843 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.362192 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.362197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.362199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.362674 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.362831 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.363232 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.363567 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":120,"server-negotiator.queue_time_us":492,"thread_start_us":300,"threads_started":1}
I20260806 18:39:28.364743 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.349962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42025 (local address 127.0.0.1:53370)
0806 18:39:28.350834 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.350909 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:28.351941 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:28.352791 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:28.352797 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:28.353260 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:28.353525 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.353540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.354746 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.354749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.355548 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.355558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.356347 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.356355 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.356566 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.357309 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:28.357330 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:28.359165 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:28.361331 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.361342 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.361358 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.361597 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.361950 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.361953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.361955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.362092 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.362812 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:28.362819 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:28.362971 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 18:39:28.363154 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 18:39:28.363513 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":33,"spinlock_wait_cycles":61824,"thread_start_us":300,"threads_started":1}
I20260806 18:39:28.367494 9872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:39:28.368386 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.368057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:45035 (local address 127.0.0.1:59706)
0806 18:39:28.368223 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.368307 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 18:39:28.368606 9817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.125.189:45035: connect: Connection refused (error 111)
I20260806 18:39:28.369683 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.369398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:45035 (local address 127.0.0.1:59710)
0806 18:39:28.369563 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.369620 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260806 18:39:28.369853 9872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.189:45035: Network error: Client connection negotiation failed: client connection to 127.18.125.189:45035: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab.
Entry for principal kudu/127.18.125.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab.
I20260806 18:39:28.373576 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.189:45035
--webserver_interface=127.18.125.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab
--principal=kudu/127.18.125.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:42025,127.18.125.189:45035,127.18.125.188:37663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:39:28.419142 9872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } attempt: 1
I20260806 18:39:28.420070 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.419752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:45035 (local address 127.0.0.1:59724)
0806 18:39:28.419915 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.419992 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 18:39:28.420871 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.420650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:45035 (local address 127.0.0.1:59740)
0806 18:39:28.420763 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.420817 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 18:39:28.421038 9872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.189:45035: Network error: Client connection negotiation failed: client connection to 127.18.125.189:45035: connect: Connection refused (error 111)
I20260806 18:39:28.489228 9872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } attempt: 2
I20260806 18:39:28.490226 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.489879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:45035 (local address 127.0.0.1:59752)
0806 18:39:28.490069 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.490144 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 18:39:28.491055 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.490813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:45035 (local address 127.0.0.1:59768)
0806 18:39:28.490927 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.490979 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:45035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 18:39:28.491231 9872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.189:45035: Network error: Client connection negotiation failed: client connection to 127.18.125.189:45035: connect: Connection refused (error 111)
W20260806 18:39:28.492565 9879 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:39:28.492918 9879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:28.492966 9879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:28.493055 9879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:28.497401 9879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:28.497476 9879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:28.497496 9879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:28.497514 9879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:28.497530 9879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:28.503543 9879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:42025,127.18.125.189:45035,127.18.125.188:37663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab
--principal=kudu/127.18.125.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.189:45035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.125.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:28.504856 9879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:28.505992 9879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:28.515637 9884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:28.515637 9887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:28.515647 9885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:28.515683 9879 server_base.cc:1034] running on GCE node
I20260806 18:39:28.516683 9879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:28.518038 9879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:28.519261 9879 hybrid_clock.cc:648] HybridClock initialized: now 1786041568519206 us; error 72 us; skew 500 ppm
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:28.522527 9879 init.cc:377] Logged in from keytab as kudu/127.18.125.189@KRBTEST.COM (short username kudu)
I20260806 18:39:28.523576 9879 webserver.cc:460] Webserver started at http://127.18.125.189:44295/ using document root <none> and password file <none>
I20260806 18:39:28.524323 9879 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:28.524431 9879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:28.524696 9879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:28.527060 9879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/data/instance:
uuid: "d611b41dfdaf449ea0a1ae27ca7baf52"
format_stamp: "Formatted at 2026-08-06 18:39:28 on dist-test-slave-8w9v"
server_key: "23cf9b322197588ea1f9fbce96e7eef8"
server_key_iv: "c666db29627ce3d4e5a7f18d96c6a3e7"
server_key_version: "encryptionkey@0"
I20260806 18:39:28.527714 9879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/wal/instance:
uuid: "d611b41dfdaf449ea0a1ae27ca7baf52"
format_stamp: "Formatted at 2026-08-06 18:39:28 on dist-test-slave-8w9v"
server_key: "23cf9b322197588ea1f9fbce96e7eef8"
server_key_iv: "c666db29627ce3d4e5a7f18d96c6a3e7"
server_key_version: "encryptionkey@0"
I20260806 18:39:28.531949 9879 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 18:39:28.534912 9895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:28.536444 9879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:28.536607 9879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/wal
uuid: "d611b41dfdaf449ea0a1ae27ca7baf52"
format_stamp: "Formatted at 2026-08-06 18:39:28 on dist-test-slave-8w9v"
server_key: "23cf9b322197588ea1f9fbce96e7eef8"
server_key_iv: "c666db29627ce3d4e5a7f18d96c6a3e7"
server_key_version: "encryptionkey@0"
I20260806 18:39:28.536751 9879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:28.548703 9879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:28.552350 9879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:28.552605 9879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:28.560560 9879 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.189:45035
I20260806 18:39:28.560997 9948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.189:45035 every 8 connection(s)
I20260806 18:39:28.561754 9879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
I20260806 18:39:28.565220 9949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:28.569808 9949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:39:28.571276 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 9879
I20260806 18:39:28.571421 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-1/wal/instance
I20260806 18:39:28.571753 18934 external_mini_cluster.cc:1428] Setting key 09e5b1180bbd72a48bd3d1e4bccdc4d2
WARNING: no policy specified for kudu/127.18.125.188@KRBTEST.COM; defaulting to no policy
I20260806 18:39:28.582414 9872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.188@KRBTEST.COM" created.
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:39:28.589846 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.572389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53386 (local address 127.18.125.190:42025)
0806 18:39:28.572593 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.572601 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.573643 (+ 1042us) server_negotiation.cc:407] Connection header received
0806 18:39:28.574717 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.574721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.574784 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.574868 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.576278 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.576840 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.578119 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.578381 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.578511 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.584469 (+ 5958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.584474 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.584477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.584509 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.586337 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.587150 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.587154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.587156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.587277 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.587779 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.587785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.587790 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.588034 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.588193 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.589522 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.589670 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 18:39:28.590480 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.572807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42025 (local address 127.0.0.1:53386)
0806 18:39:28.573438 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.573514 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:28.574488 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:28.575050 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:28.575057 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:28.575437 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:28.576063 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.576081 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.577002 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.577006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.577968 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.577979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.578573 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.578577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.578778 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.579469 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:28.579492 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:28.584273 (+ 4781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:28.586503 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.586511 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.586538 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.587021 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.587406 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.587409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.587412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.587646 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.588201 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:28.588211 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:28.588550 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 18:39:28.588769 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 18:39:28.589257 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":159,"threads_started":1}
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
I20260806 18:39:28.593196 9949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:39:28.596733 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.583078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:45035 (local address 127.0.0.1:59772)
0806 18:39:28.583620 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.583638 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:28.583744 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:28.584488 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:28.584491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:28.584734 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:28.584919 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.584926 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.586940 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.586944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.587687 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.587695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.588975 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.588980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.589126 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.589797 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:28.589820 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:28.591958 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:28.594747 (+ 2789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.594752 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.594755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.595076 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.595439 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.595445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.595450 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.595554 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.596096 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:28.596099 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:28.596242 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 18:39:28.596377 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 18:39:28.596542 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435}
I20260806 18:39:28.596840 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.583289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59772 (local address 127.18.125.189:45035)
0806 18:39:28.583799 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.583814 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.583848 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:39:28.583926 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.583936 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.584260 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.584527 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.585084 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.586801 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.587833 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.588784 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.591151 (+ 2367us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.592163 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.592176 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.592195 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.592220 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.594611 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.595213 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.595220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.595222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.595319 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.595660 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.595664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.595666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.595968 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.596123 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.596478 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.596633 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":221,"threads_started":1}
I20260806 18:39:28.599560 9872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
I20260806 18:39:28.600553 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.600222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55674)
0806 18:39:28.600384 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.600454 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 18:39:28.601302 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.601058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55676)
0806 18:39:28.601193 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.601241 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 18:39:28.601461 9872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:37663: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab.
Entry for principal kudu/127.18.125.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab.
I20260806 18:39:28.604424 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.593922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:45035 (local address 127.0.0.1:59784)
0806 18:39:28.594827 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.594843 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:28.594935 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:28.595218 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:28.595221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:28.595438 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:28.595624 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.595640 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.596641 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.596644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.597467 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.597478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.598048 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.598052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.598154 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.598715 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:28.598728 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:28.600398 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:28.602533 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.602537 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.602539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.602788 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.603184 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.603188 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.603191 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.603311 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.603887 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:28.603890 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:28.603979 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 18:39:28.604072 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:39:28.604267 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813}
I20260806 18:39:28.604614 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.594010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59784 (local address 127.18.125.189:45035)
0806 18:39:28.594433 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.594444 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.594921 (+ 477us) server_negotiation.cc:407] Connection header received
0806 18:39:28.595035 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.595037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.595080 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.595208 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.595750 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.596517 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.597591 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.597837 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.597992 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.600576 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.600579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.600581 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.600625 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.602410 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.602940 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.602943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.602944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.603047 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.603513 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.603520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.603523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.603743 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.603887 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.604201 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.604355 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":140,"threads_started":1}
I20260806 18:39:28.604609 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.188:37663
--webserver_interface=127.18.125.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab
--principal=kudu/127.18.125.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:42025,127.18.125.189:45035,127.18.125.188:37663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:39:28.605370 9949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:39:28.606221 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.605867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55692)
0806 18:39:28.606007 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.606131 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 18:39:28.606837 9900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
I20260806 18:39:28.607848 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.607548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55702)
0806 18:39:28.607698 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.607769 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 18:39:28.608048 9949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:37663: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
I20260806 18:39:28.632642 9872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } attempt: 1
I20260806 18:39:28.633666 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.633314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55712)
0806 18:39:28.633501 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.633575 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 18:39:28.634516 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.634266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55728)
0806 18:39:28.634393 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.634451 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 18:39:28.634660 9872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:37663: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
I20260806 18:39:28.657413 9949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } attempt: 1
I20260806 18:39:28.658481 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.658109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55742)
0806 18:39:28.658302 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.658379 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 18:39:28.659266 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.659030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55746)
0806 18:39:28.659149 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.659204 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 18:39:28.659435 9949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:37663: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
I20260806 18:39:28.709865 9872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } attempt: 2
I20260806 18:39:28.710858 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.710524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55748)
0806 18:39:28.710687 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.710759 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 18:39:28.711790 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.711571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55760)
0806 18:39:28.711687 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.711735 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 18:39:28.712262 9872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:37663: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
I20260806 18:39:28.727608 9949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } attempt: 2
I20260806 18:39:28.728674 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.728301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55772)
0806 18:39:28.728471 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.728571 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 18:39:28.729559 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.729270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55788)
0806 18:39:28.729405 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.729471 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 18:39:28.729700 9949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:37663: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
W20260806 18:39:28.732537 9956 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:39:28.732852 9956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:28.732882 9956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:28.732926 9956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:28.737231 9956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:28.737303 9956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:28.737324 9956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:28.737342 9956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:28.737357 9956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:28.743405 9956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:42025,127.18.125.189:45035,127.18.125.188:37663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab
--principal=kudu/127.18.125.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.188:37663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.125.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:28.744844 9956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:28.746389 9956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:28.758755 9967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:28.758926 9963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:28.758987 9956 server_base.cc:1034] running on GCE node
W20260806 18:39:28.758783 9962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:28.760272 9956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:28.761914 9956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:28.765322 9956 hybrid_clock.cc:648] HybridClock initialized: now 1786041568765235 us; error 75 us; skew 500 ppm
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:28.770042 9956 init.cc:377] Logged in from keytab as kudu/127.18.125.188@KRBTEST.COM (short username kudu)
I20260806 18:39:28.771543 9956 webserver.cc:460] Webserver started at http://127.18.125.188:42031/ using document root <none> and password file <none>
I20260806 18:39:28.772528 9956 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:28.772653 9956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:28.772984 9956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:28.776522 9956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/data/instance:
uuid: "f91b51e59ede4c6988f169f71c1bf870"
format_stamp: "Formatted at 2026-08-06 18:39:28 on dist-test-slave-8w9v"
server_key: "da57f47dfdb096612269fd7f09480703"
server_key_iv: "b6876ecc85e4ca2498894151486e353e"
server_key_version: "encryptionkey@0"
I20260806 18:39:28.777427 9956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/wal/instance:
uuid: "f91b51e59ede4c6988f169f71c1bf870"
format_stamp: "Formatted at 2026-08-06 18:39:28 on dist-test-slave-8w9v"
server_key: "da57f47dfdb096612269fd7f09480703"
server_key_iv: "b6876ecc85e4ca2498894151486e353e"
server_key_version: "encryptionkey@0"
I20260806 18:39:28.782622 9956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 18:39:28.785526 9974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:28.786940 9956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:39:28.787091 9956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/wal
uuid: "f91b51e59ede4c6988f169f71c1bf870"
format_stamp: "Formatted at 2026-08-06 18:39:28 on dist-test-slave-8w9v"
server_key: "da57f47dfdb096612269fd7f09480703"
server_key_iv: "b6876ecc85e4ca2498894151486e353e"
server_key_version: "encryptionkey@0"
I20260806 18:39:28.787235 9956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:28.806949 9956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:28.810695 9956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:28.810940 9956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:28.811467 9872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } attempt: 3
I20260806 18:39:28.812376 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.812029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55796)
0806 18:39:28.812197 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.812266 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 18:39:28.813148 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.812926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55812)
0806 18:39:28.813039 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.813094 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 18:39:28.813287 9872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:37663: Network error: Client connection negotiation failed: client connection to 127.18.125.188:37663: connect: Connection refused (error 111)
I20260806 18:39:28.820875 9949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } attempt: 3
I20260806 18:39:28.821205 9956 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.188:37663
I20260806 18:39:28.821574 10029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.188:37663 every 8 connection(s)
I20260806 18:39:28.822373 9956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
I20260806 18:39:28.823427 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 9956
I20260806 18:39:28.823568 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041269054381-18934-0/minicluster-data/master-2/wal/instance
I20260806 18:39:28.823872 18934 external_mini_cluster.cc:1428] Setting key f07dde57d79abc4b0843d75523622d29
I20260806 18:39:28.826051 10031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
I20260806 18:39:28.831470 10031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
I20260806 18:39:28.840706 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.821536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55822)
0806 18:39:28.821720 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.821739 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:28.821848 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:28.824830 (+ 2982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:28.824833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:28.825059 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:28.825190 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.825199 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.826930 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.826933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.827720 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.827734 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.828935 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.828939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.829032 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.829644 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:28.829663 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:28.831409 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:28.837999 (+ 6590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.838004 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.838007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.838319 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.838826 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.838830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.838832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.838969 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.839830 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:28.839835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:28.839956 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 18:39:28.840285 (+ 329us) client_negotiation.cc:240] Negotiation successful
0806 18:39:28.840484 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 18:39:28.842113 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.824983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55828 (local address 127.18.125.188:37663)
0806 18:39:28.827053 (+ 2070us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.827061 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.827073 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:39:28.827104 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.827106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.827145 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.827282 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.828273 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.829136 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.830531 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.830800 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.830927 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.836089 (+ 5162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.836093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.836095 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.836195 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.838346 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.838958 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.838962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.838963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.839057 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.839398 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.839402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.839403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.839757 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.840161 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.840893 (+ 732us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.841129 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1981,"spinlock_wait_cycles":15616,"thread_start_us":87,"threads_started":1}
I20260806 18:39:28.842798 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.822545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55822 (local address 127.18.125.188:37663)
0806 18:39:28.823002 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.823022 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.823074 (+ 52us) server_negotiation.cc:407] Connection header received
0806 18:39:28.823907 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.823950 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.824469 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.824871 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.825341 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.826752 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.827877 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.828771 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.828951 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.831653 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.831660 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.831680 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.831713 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.837801 (+ 6088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.838477 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.838483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.838490 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.838680 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.839092 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.839096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.839098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.839713 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.839943 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.840460 (+ 517us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.842611 (+ 2151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1810,"server-negotiator.queue_time_us":291,"thread_start_us":111,"threads_started":1}
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:39:28.847571 9949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52: Bootstrap starting.
I20260806 18:39:28.852536 9949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:28.853124 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.838404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:42025 (local address 127.0.0.1:53402)
0806 18:39:28.839070 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.839135 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:28.839313 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:28.839698 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:28.839704 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:28.840593 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:28.840826 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.840840 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.841850 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.841853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.842634 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.842646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.843259 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.843264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.843479 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.844859 (+ 1380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:28.844872 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:28.846698 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:28.850759 (+ 4061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.850776 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.850784 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.851155 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.851541 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.851548 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.851552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.851661 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.852298 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:28.852305 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:28.852450 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 18:39:28.852708 (+ 258us) client_negotiation.cc:240] Negotiation successful
0806 18:39:28.852970 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":6,"thread_start_us":186,"threads_started":1}
I20260806 18:39:28.853652 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.838570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53402 (local address 127.18.125.190:42025)
0806 18:39:28.838750 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.838755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.839214 (+ 459us) server_negotiation.cc:407] Connection header received
0806 18:39:28.839481 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.839485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.839548 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.839639 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.841009 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.841710 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.842791 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.843109 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.843261 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.847070 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.847083 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.847085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.847353 (+ 268us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.850594 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.851285 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.851289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.851290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.851406 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.851789 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.851793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.851795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.852090 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.852280 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.853398 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.853540 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 18:39:28.853807 9949 log.cc:824] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:28.854836 10031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:39:28.857649 9949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52: No bootstrap required, opened a new log
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
I20260806 18:39:28.862066 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.849990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.18.125.190:42025)
0806 18:39:28.850510 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.850527 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.850698 (+ 171us) server_negotiation.cc:407] Connection header received
0806 18:39:28.850750 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.850753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.850806 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.851064 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.851889 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.852726 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.853966 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.854320 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.854422 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.857553 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.857557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.857560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.857587 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.859774 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.860406 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.860411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.860412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.860524 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.860883 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.860886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.860888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.861187 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.861432 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.861648 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.861797 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":310,"thread_start_us":130,"threads_started":1}
I20260806 18:39:28.862960 9949 raft_consensus.cc:348] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } }
I20260806 18:39:28.863217 9949 raft_consensus.cc:729] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d611b41dfdaf449ea0a1ae27ca7baf52, State: Initialized, Role: FOLLOWER
I20260806 18:39:28.863586 9949 consensus_queue.cc:260] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } }
I20260806 18:39:28.864362 10039 sys_catalog.cc:455] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } } }
I20260806 18:39:28.864548 10039 sys_catalog.cc:458] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:28.865437 9949 sys_catalog.cc:564] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:28.865906 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.855596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59786 (local address 127.18.125.189:45035)
0806 18:39:28.855756 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.855780 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.855797 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:39:28.855850 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.855853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.855915 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.856022 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.856736 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.857580 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.858834 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.859086 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.859211 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.861438 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.861442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.861444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.861474 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.863499 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.864253 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.864256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.864258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.864351 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.864846 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.864851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.864854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.865145 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.865252 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.865614 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.865741 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:39:28.865936 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.855492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:45035 (local address 127.0.0.1:59786)
0806 18:39:28.855619 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.855636 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:28.855718 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:28.856091 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:28.856094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:28.856411 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:28.856553 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.856563 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.857698 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.857702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.858495 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.858507 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.859228 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.859232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.859333 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.859806 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:28.859824 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:28.861235 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:28.863630 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.863634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.863636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.864059 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.864473 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.864476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.864478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.864696 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.865268 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:28.865271 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:28.865384 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 18:39:28.865565 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 18:39:28.865735 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 18:39:28.866997 10031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
W20260806 18:39:28.872030 10053 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:28.872148 10053 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:28.870469 9879 master_runner.cc:186] Error getting master registration for 127.18.125.190:42025: OK, instance_id { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" instance_seqno: 1786041568335625 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:39:28.873936 9879 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
I20260806 18:39:28.875361 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.863961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.18.125.189:45035)
0806 18:39:28.864126 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.864153 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.864169 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:39:28.864207 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.864211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.864258 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.864346 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.865262 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.865990 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.866970 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.867394 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.867500 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.870670 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.870674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.870676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.870701 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.872770 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.873313 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.873317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.873319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.873416 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.873744 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.873748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.873751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.874254 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.874472 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.874847 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.875234 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":57}
I20260806 18:39:28.878463 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.867642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55832)
0806 18:39:28.868126 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.868169 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:28.868291 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:28.868658 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:28.868664 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:28.868855 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:28.868972 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.868981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.870066 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.870071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.870867 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.870881 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.872328 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.872333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.872429 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.872927 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:28.872941 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:28.874467 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:28.876624 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.876628 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.876631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.876887 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.877209 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.877212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.877214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.877322 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.877835 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:28.877839 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:28.877951 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 18:39:28.878085 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 18:39:28.878254 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 18:39:28.878520 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.867861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.18.125.188:37663)
0806 18:39:28.867966 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.867970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.868223 (+ 253us) server_negotiation.cc:407] Connection header received
0806 18:39:28.868412 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.868415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.868462 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.868511 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.869104 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.869930 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.871343 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.872034 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.872152 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.874639 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.874643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.874646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.874672 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.876505 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.877007 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.877014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.877016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.877107 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.877436 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.877439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.877441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.877722 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.877830 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.878163 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.878334 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":35}
I20260806 18:39:28.880055 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:39:28.882205 10031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870: Bootstrap starting.
I20260806 18:39:28.881495 9843 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7c8ae76b9fac444ca80983ebd6f594d7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53410
I20260806 18:39:28.885535 10031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:28.886565 10031 log.cc:824] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:28.889380 10031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870: No bootstrap required, opened a new log
I20260806 18:39:28.894986 10031 raft_consensus.cc:348] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } }
I20260806 18:39:28.895351 10031 raft_consensus.cc:729] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f91b51e59ede4c6988f169f71c1bf870, State: Initialized, Role: FOLLOWER
I20260806 18:39:28.895744 10031 consensus_queue.cc:260] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } }
I20260806 18:39:28.896486 10056 sys_catalog.cc:455] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } } }
I20260806 18:39:28.896664 10056 sys_catalog.cc:458] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:28.897523 10031 sys_catalog.cc:564] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 18:39:28.904011 10067 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:28.904155 10067 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:28.902280 9956 master_runner.cc:186] Error getting master registration for 127.18.125.190:42025: OK, instance_id { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" instance_seqno: 1786041568335625 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:39:28.905568 9956 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:39:28.977474 9872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } attempt: 4
Aug 06 18:39:28 dist-test-slave-8w9v krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
I20260806 18:39:28.987788 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.978224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:37663 (local address 127.0.0.1:55848)
0806 18:39:28.978382 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:28.978402 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:28.978518 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:28.978921 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:28.978924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:28.979149 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:28.979246 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.979252 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.980370 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.980374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.980932 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:28.980941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.981579 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.981583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:28.981676 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.982298 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:28.982311 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:28.983969 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:28.986012 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.986017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.986020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.986291 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.986592 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:28.986596 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:28.986598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:28.986699 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:28.987168 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:28.987173 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:28.987278 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 18:39:28.987419 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 18:39:28.987584 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 18:39:28.987788 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:28.978521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55848 (local address 127.18.125.188:37663)
0806 18:39:28.978663 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:28.978670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:28.978688 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:39:28.978741 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:28.978747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:28.978796 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:28.978884 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:28.979400 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.980250 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.981081 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:28.981447 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:28.981563 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:28.984175 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:28.984181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:28.984188 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:28.984217 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:28.985864 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.986386 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.986390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.986392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.986484 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:28.986807 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:28.986810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:28.986812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:28.987061 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:28.987154 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:28.987481 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 18:39:28.987605 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:39:28.992025 9872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7: Bootstrap starting.
I20260806 18:39:28.995198 9872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:28.996209 9872 log.cc:824] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:28.999006 9872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7: No bootstrap required, opened a new log
I20260806 18:39:29.004688 9872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } }
I20260806 18:39:29.005720 9872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c8ae76b9fac444ca80983ebd6f594d7, State: Initialized, Role: FOLLOWER
I20260806 18:39:29.006191 9872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } }
I20260806 18:39:29.006922 10071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } } }
I20260806 18:39:29.007225 10071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:29.008013 9872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:29.008052 9843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 18:39:29.008214 9843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:29.010529 9843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } }
I20260806 18:39:29.012451 9923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52"
I20260806 18:39:29.012497 10004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f91b51e59ede4c6988f169f71c1bf870"
I20260806 18:39:29.012755 10004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:29.012755 9923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:29.014240 9923 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c8ae76b9fac444ca80983ebd6f594d7 in term 1.
I20260806 18:39:29.014233 10004 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c8ae76b9fac444ca80983ebd6f594d7 in term 1.
I20260806 18:39:29.014914 9819 leader_election.cc:302] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c8ae76b9fac444ca80983ebd6f594d7, f91b51e59ede4c6988f169f71c1bf870; no voters:
W20260806 18:39:29.018419 10082 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:29.018519 10082 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:29.019101 9843 leader_election.cc:288] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [CANDIDATE]: Term 1 election: Requested vote from peers d611b41dfdaf449ea0a1ae27ca7baf52 (127.18.125.189:45035), f91b51e59ede4c6988f169f71c1bf870 (127.18.125.188:37663)
I20260806 18:39:29.021462 9799 master_runner.cc:263] No leader master found from master 7c8ae76b9fac444ca80983ebd6f594d7
I20260806 18:39:29.021543 9799 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:39:29.021770 10071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:29.022130 10071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [term 1 LEADER]: Becoming Leader. State: Replica: 7c8ae76b9fac444ca80983ebd6f594d7, State: Running, Role: LEADER
I20260806 18:39:29.022925 10071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } }
I20260806 18:39:29.025645 10088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c8ae76b9fac444ca80983ebd6f594d7. Latest consensus state: current_term: 1 leader_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } } }
I20260806 18:39:29.027271 10088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:29.037545 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:29.022322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.18.125.189:45035)
0806 18:39:29.022463 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.022468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.025777 (+ 3309us) server_negotiation.cc:407] Connection header received
0806 18:39:29.025947 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.025953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.026017 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.026099 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.027626 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:29.028565 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:29.029868 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:29.030236 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:29.030331 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:29.032988 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:29.032993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:29.032996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:29.033033 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:29.035216 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:29.035848 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:29.035853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:29.035856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:29.036009 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:29.036403 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:29.036407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:29.036409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:29.036666 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:29.036796 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:29.037201 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 18:39:29.037350 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:39:29.039903 10094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:29.041765 10094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:29.051092 10004 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [term 1 FOLLOWER]: Refusing update from remote peer 7c8ae76b9fac444ca80983ebd6f594d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:29.051962 10088 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:39:29.054566 9902 tablet.cc:2369] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:39:29.057832 10056 sys_catalog.cc:455] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c8ae76b9fac444ca80983ebd6f594d7. Latest consensus state: current_term: 1 leader_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } } }
I20260806 18:39:29.058001 10056 sys_catalog.cc:458] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:29.058073 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:29.024566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53424 (local address 127.18.125.190:42025)
0806 18:39:29.024723 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.024729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.032485 (+ 7756us) server_negotiation.cc:407] Connection header received
0806 18:39:29.032988 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.032992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.033052 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.033108 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.035270 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:29.036244 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:29.039425 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:29.039849 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:29.039942 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:29.044408 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:29.044413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:29.044417 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:29.044455 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:29.046342 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:29.050299 (+ 3957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:29.050304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:29.050306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:29.050436 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:29.055292 (+ 4856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:29.055298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:29.055300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:29.055663 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:29.055816 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:29.057671 (+ 1855us) server_negotiation.cc:299] Negotiation successful
0806 18:39:29.057862 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:39:29.059473 9923 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [term 1 FOLLOWER]: Refusing update from remote peer 7c8ae76b9fac444ca80983ebd6f594d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:29.060312 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:29.025105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55858 (local address 127.18.125.188:37663)
0806 18:39:29.025242 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.025249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.032892 (+ 7643us) server_negotiation.cc:407] Connection header received
0806 18:39:29.034197 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.034201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.034259 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.034319 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.037491 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:29.038252 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:29.040906 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:29.041255 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:29.041346 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:29.047014 (+ 5668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:29.047018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:29.047021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:29.047058 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:29.049461 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:29.056330 (+ 6869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:29.056334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:29.056336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:29.056472 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:29.056839 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:29.056842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:29.056844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:29.057093 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:29.057238 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:29.059913 (+ 2675us) server_negotiation.cc:299] Negotiation successful
0806 18:39:29.060087 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:39:29.060300 10088 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:39:29.064311 9981 tablet.cc:2369] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:39:29.066979 10039 sys_catalog.cc:455] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c8ae76b9fac444ca80983ebd6f594d7. Latest consensus state: current_term: 1 leader_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } } }
I20260806 18:39:29.067166 10039 sys_catalog.cc:458] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:29.076500 10088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } } }
I20260806 18:39:29.077280 10088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:29.078956 10039 sys_catalog.cc:455] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } } }
I20260806 18:39:29.079113 10039 sys_catalog.cc:458] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:29.079628 10095 mvcc.cc:205] Tried to move back new op lower bound from 7315626266821726208 to 7315626266721763328. Current Snapshot: MvccSnapshot[applied={T|T < 7315626266821726208}]
I20260806 18:39:29.080900 10071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } } }
I20260806 18:39:29.081063 10071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:29.082494 10056 sys_catalog.cc:455] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c8ae76b9fac444ca80983ebd6f594d7" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 42025 } } peers { permanent_uuid: "d611b41dfdaf449ea0a1ae27ca7baf52" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 45035 } } peers { permanent_uuid: "f91b51e59ede4c6988f169f71c1bf870" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 37663 } } }
I20260806 18:39:29.082633 10056 sys_catalog.cc:458] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:29.083518 10098 mvcc.cc:205] Tried to move back new op lower bound from 7315626266821726208 to 7315626266721763328. Current Snapshot: MvccSnapshot[applied={T|T < 7315626266821726208}]
I20260806 18:39:29.083732 10096 mvcc.cc:205] Tried to move back new op lower bound from 7315626266821726208 to 7315626266721763328. Current Snapshot: MvccSnapshot[applied={T|T < 7315626266821726208}]
I20260806 18:39:29.090044 10094 catalog_manager.cc:1347] Generated new cluster ID: 702df72a0a1c46ffa42dbc23f9e9bdf9
I20260806 18:39:29.090162 10094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:29.129302 10094 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:29.130903 10094 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:29.154397 10094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c8ae76b9fac444ca80983ebd6f594d7: Generated new TSK 0
I20260806 18:39:29.155411 10094 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:39:29.189764 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.188074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.18.125.189:45035)
0806 18:39:29.188381 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.188386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.188402 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:39:29.188561 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.188565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.188652 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.188719 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.189588 (+ 869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59818: BlockingRecv error: recv got EOF from 127.0.0.1:59818 (error 108)
Metrics: {"server-negotiator.queue_time_us":226}
W20260806 18:39:29.189790 10036 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.187930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.18.125.190:42025)
0806 18:39:29.188270 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.188275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.188685 (+ 410us) server_negotiation.cc:407] Connection header received
0806 18:39:29.188880 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.188883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.188940 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.188992 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.189652 (+ 660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53436: BlockingRecv error: recv got EOF from 127.0.0.1:53436 (error 108)
Metrics: {"server-negotiator.queue_time_us":223}
W20260806 18:39:29.190634 10030 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.188442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55860 (local address 127.18.125.188:37663)
0806 18:39:29.188588 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.188593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.189113 (+ 520us) server_negotiation.cc:407] Connection header received
0806 18:39:29.189336 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.189340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.189400 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.189461 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.190494 (+ 1033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55860: BlockingRecv error: recv got EOF from 127.0.0.1:55860 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 18:39:29.194319 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.193094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59826 (local address 127.18.125.189:45035)
0806 18:39:29.193222 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.193227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.193347 (+ 120us) server_negotiation.cc:407] Connection header received
0806 18:39:29.193564 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.193568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.193643 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.193741 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.194201 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59826: BlockingRecv error: recv got EOF from 127.0.0.1:59826 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 18:39:29.199265 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.198196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59828 (local address 127.18.125.189:45035)
0806 18:39:29.198328 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.198331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.198343 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:39:29.198390 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.198392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.198436 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.198502 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.199131 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59828: BlockingRecv error: recv got EOF from 127.0.0.1:59828 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 18:39:29.204317 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.203177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.18.125.189:45035)
0806 18:39:29.203335 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.203340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.203449 (+ 109us) server_negotiation.cc:407] Connection header received
0806 18:39:29.203550 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.203554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.203615 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.203721 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.204177 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59832: BlockingRecv error: recv got EOF from 127.0.0.1:59832 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 18:39:29.211666 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.210337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.18.125.189:45035)
0806 18:39:29.210518 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.210523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.210706 (+ 183us) server_negotiation.cc:407] Connection header received
0806 18:39:29.210912 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.210916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.210975 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.211066 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.211544 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59846: BlockingRecv error: recv got EOF from 127.0.0.1:59846 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260806 18:39:29.223268 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.222049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59850 (local address 127.18.125.189:45035)
0806 18:39:29.222223 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.222228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.222348 (+ 120us) server_negotiation.cc:407] Connection header received
0806 18:39:29.222442 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.222446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.222508 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.222625 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.223131 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59850: BlockingRecv error: recv got EOF from 127.0.0.1:59850 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 18:39:29.244088 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.242950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59860 (local address 127.18.125.189:45035)
0806 18:39:29.243128 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.243132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.243270 (+ 138us) server_negotiation.cc:407] Connection header received
0806 18:39:29.243348 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.243351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.243408 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.243510 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.243968 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59860: BlockingRecv error: recv got EOF from 127.0.0.1:59860 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 18:39:29.279975 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.278762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59862 (local address 127.18.125.189:45035)
0806 18:39:29.278932 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.278936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.279086 (+ 150us) server_negotiation.cc:407] Connection header received
0806 18:39:29.279173 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.279176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.279229 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.279317 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.279803 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59862: BlockingRecv error: recv got EOF from 127.0.0.1:59862 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 18:39:29.353432 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.352085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59878 (local address 127.18.125.189:45035)
0806 18:39:29.352261 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.352267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.352474 (+ 207us) server_negotiation.cc:407] Connection header received
0806 18:39:29.352658 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.352661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.352713 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.352811 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.353289 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59878: BlockingRecv error: recv got EOF from 127.0.0.1:59878 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 18:39:29.485854 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.484613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59892 (local address 127.18.125.189:45035)
0806 18:39:29.484764 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.484768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.484976 (+ 208us) server_negotiation.cc:407] Connection header received
0806 18:39:29.485060 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.485062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.485117 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.485214 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.485749 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59892: BlockingRecv error: recv got EOF from 127.0.0.1:59892 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 18:39:29.745808 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:29.744602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59904 (local address 127.18.125.189:45035)
0806 18:39:29.744768 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:29.744775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:29.744793 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:39:29.745000 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:29.745003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:29.745055 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:29.745153 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:29.745657 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59904: BlockingRecv error: recv got EOF from 127.0.0.1:59904 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 18:39:29.873593 10053 catalog_manager.cc:1259] Loaded cluster ID: 702df72a0a1c46ffa42dbc23f9e9bdf9
I20260806 18:39:29.873697 10053 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52: loading cluster ID for follower catalog manager: success
I20260806 18:39:29.875475 10053 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52: acquiring CA information for follower catalog manager: success
I20260806 18:39:29.876753 10053 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d611b41dfdaf449ea0a1ae27ca7baf52: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:39:29.905592 10067 catalog_manager.cc:1259] Loaded cluster ID: 702df72a0a1c46ffa42dbc23f9e9bdf9
I20260806 18:39:29.905725 10067 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870: loading cluster ID for follower catalog manager: success
I20260806 18:39:29.907464 10067 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870: acquiring CA information for follower catalog manager: success
I20260806 18:39:29.908857 10067 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f91b51e59ede4c6988f169f71c1bf870: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:39:30.267014 10173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:30.261443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59920 (local address 127.18.125.189:45035)
0806 18:39:30.261782 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:30.261786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:30.261808 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:39:30.261881 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:30.261884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:30.262035 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:30.262161 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:30.262571 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:30.263322 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:30.264525 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:30.264790 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:30.264891 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:30.265536 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:30.266183 (+ 647us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:30.266318 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:30.266688 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 18:39:30.266764 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":118,"threads_started":1}
I20260806 18:39:30.276001 10179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:30.271321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55872 (local address 127.18.125.188:37663)
0806 18:39:30.271634 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:30.271638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:30.271795 (+ 157us) server_negotiation.cc:407] Connection header received
0806 18:39:30.271961 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:30.271964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:30.272094 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:30.272269 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:30.272521 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:30.273087 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:30.274008 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:30.274244 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:30.274343 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:30.274935 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:30.275441 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:30.275545 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:30.275714 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 18:39:30.275774 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":87,"threads_started":1}
I20260806 18:39:30.285009 10173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:30.280579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59924 (local address 127.18.125.189:45035)
0806 18:39:30.280740 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:30.280744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:30.281048 (+ 304us) server_negotiation.cc:407] Connection header received
0806 18:39:30.281266 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:30.281269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:30.281321 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:30.281419 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:30.281684 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:30.282407 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:30.283502 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:30.283757 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:30.283872 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:30.284433 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:30.284525 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:30.284572 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:30.284760 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:39:30.284838 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:39:30.285605 10179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:30.281017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55888 (local address 127.18.125.188:37663)
0806 18:39:30.281136 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:30.281140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:30.281367 (+ 227us) server_negotiation.cc:407] Connection header received
0806 18:39:30.281510 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:30.281514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:30.281569 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:30.281633 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:30.282013 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:30.282711 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:30.283857 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:30.284233 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:30.284354 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:30.284617 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:30.284719 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:30.284791 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:30.285386 (+ 595us) server_negotiation.cc:299] Negotiation successful
0806 18:39:30.285482 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:39:30.286257 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:30.280639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53446 (local address 127.18.125.190:42025)
0806 18:39:30.280918 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:30.280926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:30.281279 (+ 353us) server_negotiation.cc:407] Connection header received
0806 18:39:30.281583 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:30.281589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:30.281771 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:30.281877 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:30.282236 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:30.283006 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:30.284306 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:30.284690 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:30.284789 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:30.285204 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:30.285571 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:30.285686 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:30.285984 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 18:39:30.286051 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":74,"threads_started":1}
I20260806 18:39:30.289819 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 9799
I20260806 18:39:30.298373 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 9879
I20260806 18:39:30.305896 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 9956
2026-08-06T18:39:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3408 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:30 dist-test-slave-8w9v krb5kdc[10191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:30 dist-test-slave-8w9v krb5kdc[10191](info): set up 2 sockets
Aug 06 18:39:30 dist-test-slave-8w9v krb5kdc[10191](info): commencing operation
krb5kdc: starting...
W20260806 18:39:31.396303 18934 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/test-user.keytab.
Aug 06 18:39:31 dist-test-slave-8w9v krb5kdc[10191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.125.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
Entry for principal kudu/127.18.125.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab.
I20260806 18:39:31.536597 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:33823
--webserver_interface=127.18.125.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:33823,127.18.125.189:44967,127.18.125.188:38587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:31.657018 10205 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:39:31.657440 10205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:31.657516 10205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:31.657614 10205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:31.662233 10205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:31.662356 10205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:31.662402 10205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:31.662446 10205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:31.662487 10205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:31.668190 10205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:33823,127.18.125.189:44967,127.18.125.188:38587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.190.keytab
--principal=kudu/127.18.125.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.190:33823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.125.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:31.669443 10205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:31.670563 10205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:31.679917 10211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:31.679921 10214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:31.680016 10205 server_base.cc:1034] running on GCE node
W20260806 18:39:31.679921 10210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:31.680802 10205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:31.682142 10205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:31.683333 10205 hybrid_clock.cc:648] HybridClock initialized: now 1786041571683301 us; error 43 us; skew 500 ppm
Aug 06 18:39:31 dist-test-slave-8w9v krb5kdc[10191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:31.686383 10205 init.cc:377] Logged in from keytab as kudu/127.18.125.190@KRBTEST.COM (short username kudu)
I20260806 18:39:31.687369 10205 webserver.cc:460] Webserver started at http://127.18.125.190:46319/ using document root <none> and password file <none>
I20260806 18:39:31.688005 10205 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:31.688108 10205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:31.688367 10205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:31.690626 10205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/data/instance:
uuid: "afd4c5d989d844edbb306421897589b1"
format_stamp: "Formatted at 2026-08-06 18:39:31 on dist-test-slave-8w9v"
server_key: "c0febeee798b81032d61bd4f5d2b6e4a"
server_key_iv: "3d7e027a2dc6d2072074a8e3b2654606"
server_key_version: "encryptionkey@0"
I20260806 18:39:31.691238 10205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/wal/instance:
uuid: "afd4c5d989d844edbb306421897589b1"
format_stamp: "Formatted at 2026-08-06 18:39:31 on dist-test-slave-8w9v"
server_key: "c0febeee798b81032d61bd4f5d2b6e4a"
server_key_iv: "3d7e027a2dc6d2072074a8e3b2654606"
server_key_version: "encryptionkey@0"
I20260806 18:39:31.695421 10205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 18:39:31.698371 10223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:31.699877 10205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 18:39:31.700197 10205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/wal
uuid: "afd4c5d989d844edbb306421897589b1"
format_stamp: "Formatted at 2026-08-06 18:39:31 on dist-test-slave-8w9v"
server_key: "c0febeee798b81032d61bd4f5d2b6e4a"
server_key_iv: "3d7e027a2dc6d2072074a8e3b2654606"
server_key_version: "encryptionkey@0"
I20260806 18:39:31.700343 10205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:31.719926 10205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:31.723466 10205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:31.723714 10205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:31.731969 10205 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.190:33823
I20260806 18:39:31.732322 10277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.190:33823 every 8 connection(s)
I20260806 18:39:31.733121 10205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:31.734370 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 10205
I20260806 18:39:31.734529 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-0/wal/instance
I20260806 18:39:31.734826 18934 external_mini_cluster.cc:1428] Setting key ead494c453a1ab29074b976577014460
I20260806 18:39:31.736441 10278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:31.742759 10278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.125.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.189@KRBTEST.COM" created.
Aug 06 18:39:31 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:39:31.764696 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:31.746073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:33823 (local address 127.0.0.1:57430)
0806 18:39:31.746813 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:31.746872 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:31.748802 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:31.749907 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:31.749917 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:31.750492 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:31.750812 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:31.750836 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:31.752505 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:31.752512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:31.753843 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:31.753860 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:31.754974 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:31.754977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:31.755067 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:31.755800 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:31.755818 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:31.757490 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:31.760781 (+ 3291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:31.760791 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:31.760801 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:31.761103 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:31.761594 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:31.761597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:31.761599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:31.761747 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:31.762567 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:31.762573 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:31.762718 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 18:39:31.762912 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 18:39:31.763435 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":27,"spinlock_wait_cycles":52864,"thread_start_us":228,"threads_started":1}
I20260806 18:39:31.764696 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:31.746074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57430 (local address 127.18.125.190:33823)
0806 18:39:31.747955 (+ 1881us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:31.747969 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:31.748021 (+ 52us) server_negotiation.cc:407] Connection header received
0806 18:39:31.749331 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:31.749337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:31.749607 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:31.749835 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:31.751015 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:31.752275 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:31.754016 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:31.754798 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:31.755019 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:31.757685 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:31.757694 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:31.757714 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:31.757746 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:31.760618 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:31.761261 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:31.761268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:31.761270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:31.761455 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:31.761878 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:31.761881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:31.761883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:31.762430 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:31.762589 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:31.763093 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 18:39:31.763495 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":206,"server-negotiator.queue_time_us":1726,"spinlock_wait_cycles":1280,"thread_start_us":217,"threads_started":1}
I20260806 18:39:31.767881 10278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:39:31.769215 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:31.768804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:44967 (local address 127.0.0.1:46472)
0806 18:39:31.768957 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:31.769130 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 18:39:31.769505 10224 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
I20260806 18:39:31.770597 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:31.770356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:44967 (local address 127.0.0.1:46484)
0806 18:39:31.770484 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:31.770535 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 18:39:31.770767 10278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.189:44967: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab.
Entry for principal kudu/127.18.125.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab.
I20260806 18:39:31.772722 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.189:44967
--webserver_interface=127.18.125.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab
--principal=kudu/127.18.125.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:33823,127.18.125.189:44967,127.18.125.188:38587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:39:31.820068 10278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } attempt: 1
I20260806 18:39:31.821113 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:31.820742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:44967 (local address 127.0.0.1:46494)
0806 18:39:31.820938 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:31.821018 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 18:39:31.821931 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:31.821663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:44967 (local address 127.0.0.1:46500)
0806 18:39:31.821806 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:31.821863 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 18:39:31.822093 10278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.189:44967: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
I20260806 18:39:31.890277 10278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } attempt: 2
I20260806 18:39:31.891237 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:31.890859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:44967 (local address 127.0.0.1:46510)
0806 18:39:31.891066 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:31.891142 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 18:39:31.892010 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:31.891772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:44967 (local address 127.0.0.1:46512)
0806 18:39:31.891890 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:31.891943 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:39:31.892175 10278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.189:44967: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
W20260806 18:39:31.894173 10285 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:39:31.894500 10285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:31.894547 10285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:31.894611 10285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:31.898907 10285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:31.898977 10285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:31.898998 10285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:31.899066 10285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:31.899093 10285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:31.905076 10285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:33823,127.18.125.189:44967,127.18.125.188:38587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.189.keytab
--principal=kudu/127.18.125.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.189:44967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.125.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:31.906253 10285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:31.907354 10285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:31.916718 10293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:31.916741 10291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:31.916741 10290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:31.917168 10285 server_base.cc:1034] running on GCE node
I20260806 18:39:31.917771 10285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:31.918972 10285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:31.920190 10285 hybrid_clock.cc:648] HybridClock initialized: now 1786041571920165 us; error 40 us; skew 500 ppm
Aug 06 18:39:31 dist-test-slave-8w9v krb5kdc[10191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:31.923378 10285 init.cc:377] Logged in from keytab as kudu/127.18.125.189@KRBTEST.COM (short username kudu)
I20260806 18:39:31.924539 10285 webserver.cc:460] Webserver started at http://127.18.125.189:43401/ using document root <none> and password file <none>
I20260806 18:39:31.925168 10285 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:31.925238 10285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:31.925474 10285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:31.927747 10285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/data/instance:
uuid: "43f5c40f21044cb0a8eb76b214eceac7"
format_stamp: "Formatted at 2026-08-06 18:39:31 on dist-test-slave-8w9v"
server_key: "7edfa97f6664a8cfc33f9d2f10d9f430"
server_key_iv: "c0631e3168dc1c97b0aafa1bb0dcc738"
server_key_version: "encryptionkey@0"
I20260806 18:39:31.928395 10285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/wal/instance:
uuid: "43f5c40f21044cb0a8eb76b214eceac7"
format_stamp: "Formatted at 2026-08-06 18:39:31 on dist-test-slave-8w9v"
server_key: "7edfa97f6664a8cfc33f9d2f10d9f430"
server_key_iv: "c0631e3168dc1c97b0aafa1bb0dcc738"
server_key_version: "encryptionkey@0"
I20260806 18:39:31.932632 10285 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:39:31.935245 10300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:31.936609 10285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:31.936775 10285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/wal
uuid: "43f5c40f21044cb0a8eb76b214eceac7"
format_stamp: "Formatted at 2026-08-06 18:39:31 on dist-test-slave-8w9v"
server_key: "7edfa97f6664a8cfc33f9d2f10d9f430"
server_key_iv: "c0631e3168dc1c97b0aafa1bb0dcc738"
server_key_version: "encryptionkey@0"
I20260806 18:39:31.936913 10285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:31.975610 10285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:31.978963 10285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:31.979204 10285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:31.983383 10278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } attempt: 3
I20260806 18:39:31.984349 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:31.984013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:44967 (local address 127.0.0.1:46518)
0806 18:39:31.984193 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:31.984268 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 18:39:31.985278 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:31.984970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:44967 (local address 127.0.0.1:46530)
0806 18:39:31.985127 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:31.985190 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 18:39:31.985453 10278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.189:44967: Network error: Client connection negotiation failed: client connection to 127.18.125.189:44967: connect: Connection refused (error 111)
I20260806 18:39:31.987672 10285 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.189:44967
I20260806 18:39:31.988006 10356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.189:44967 every 8 connection(s)
I20260806 18:39:31.988788 10285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/data/info.pb
I20260806 18:39:31.991050 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 10285
I20260806 18:39:31.991189 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-1/wal/instance
I20260806 18:39:31.991537 18934 external_mini_cluster.cc:1428] Setting key 54f583554c4e82e5e915b7053af3de1a
I20260806 18:39:31.992311 10357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:31.997051 10357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.125.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.125.188@KRBTEST.COM" created.
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:39:32.016402 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:31.999524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57440 (local address 127.18.125.190:33823)
0806 18:39:31.999742 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:31.999749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.001705 (+ 1956us) server_negotiation.cc:407] Connection header received
0806 18:39:32.002793 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.002797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.002853 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.003015 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.004822 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.005382 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.006882 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.007318 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.007439 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.010532 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.010537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.010542 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.010584 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.013299 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.014000 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.014008 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.014012 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.014120 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.014659 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.014662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.014664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.014990 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.015119 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.015975 (+ 856us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.016210 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:39:32.017587 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.000029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:33823 (local address 127.0.0.1:57440)
0806 18:39:32.001477 (+ 1448us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.001529 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:32.002531 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:32.003379 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:32.003387 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:32.003785 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:32.004610 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.004634 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.005539 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.005545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.006730 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.006743 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.007507 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.007510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.007672 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.008397 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:32.008417 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:32.010284 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:32.013477 (+ 3193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.013485 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.013503 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.013819 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.014254 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.014257 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.014259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.014499 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.015164 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:32.015173 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:32.015623 (+ 450us) client_negotiation.cc:769] Sending connection context
0806 18:39:32.015879 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 18:39:32.016327 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1193,"thread_start_us":194,"threads_started":1}
I20260806 18:39:32.019407 10357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.125.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab.
Entry for principal kudu/127.18.125.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab.
I20260806 18:39:32.024896 18934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskun4LBX/build/debug/bin/kudu
/tmp/dist-test-taskun4LBX/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.188:38587
--webserver_interface=127.18.125.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab
--principal=kudu/127.18.125.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.125.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.125.190:33823,127.18.125.189:44967,127.18.125.188:38587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
I20260806 18:39:32.030334 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.019988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:44967 (local address 127.0.0.1:46534)
0806 18:39:32.020144 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.020161 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:32.020251 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:32.020835 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:32.020838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:32.021029 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:32.021146 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.021152 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.022206 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.022209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.022775 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.022784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.023541 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.023544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.023632 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.024285 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:32.024301 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:32.025966 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:32.028528 (+ 2562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.028531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.028533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.028784 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.029108 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.029110 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.029112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.029192 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.029877 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:32.029882 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:32.029990 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 18:39:32.030079 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 18:39:32.030201 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 18:39:32.030694 10363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.020151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46534 (local address 127.18.125.189:44967)
0806 18:39:32.020466 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.020486 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.020509 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:39:32.020554 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.020558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.020713 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.020823 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.021258 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.022079 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.022956 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.023415 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.023542 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.026151 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.026168 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.026191 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.026226 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.028383 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.028888 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.028898 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.028903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.029009 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.029288 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.029294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.029298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.029720 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.029909 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.030217 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.030409 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":123,"threads_started":1}
I20260806 18:39:32.032917 10357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:39:32.033867 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.033488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:43964)
0806 18:39:32.033706 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.033791 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 18:39:32.034094 10302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
I20260806 18:39:32.034996 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.034772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:43972)
0806 18:39:32.034886 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.034942 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:39:32.035141 10357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:38587: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
I20260806 18:39:32.084450 10357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } attempt: 1
I20260806 18:39:32.085470 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.085134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:43988)
0806 18:39:32.085305 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.085385 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 18:39:32.086277 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.086038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:43996)
0806 18:39:32.086156 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.086214 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 18:39:32.086429 10357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:38587: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
I20260806 18:39:32.128645 10278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } attempt: 4
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
I20260806 18:39:32.139207 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.129350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:44967 (local address 127.0.0.1:46546)
0806 18:39:32.129587 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.129603 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:32.129695 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:32.129993 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:32.129995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:32.130236 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:32.130376 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.130382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.132225 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.132228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.132797 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.132806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.133446 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.133449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.133537 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.134126 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:32.134139 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:32.135452 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:32.137661 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.137664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.137666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.137879 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.138182 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.138184 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.138186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.138264 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.138753 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:32.138755 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:32.138856 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:39:32.138950 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:39:32.139072 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 18:39:32.139333 10363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.129466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46546 (local address 127.18.125.189:44967)
0806 18:39:32.129656 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.129661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.129682 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:39:32.129804 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.129808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.129867 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.129961 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.130505 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.132055 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.132934 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.133308 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.133434 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.135628 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.135633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.135636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.135668 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.137530 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.137984 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.137988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.137990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.138084 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.138380 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.138383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.138385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.138626 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.138736 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.139029 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.139168 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 18:39:32.140384 10278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:39:32.141150 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.140898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:43998)
0806 18:39:32.141022 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.141078 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 18:39:32.142009 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.141780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:44012)
0806 18:39:32.141902 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.141949 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 18:39:32.142222 10278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:38587: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
W20260806 18:39:32.149750 10364 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:39:32.150080 10364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:32.150133 10364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:32.150206 10364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:32.154618 10364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260806 18:39:32.154610 10357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } attempt: 2
W20260806 18:39:32.154717 10364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:32.154755 10364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:32.154798 10364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:32.154850 10364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:32.155501 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.155189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:44022)
0806 18:39:32.155345 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.155424 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 18:39:32.156303 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.156046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:44026)
0806 18:39:32.156189 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.156245 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:39:32.156435 10357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:38587: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
I20260806 18:39:32.160962 10364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.125.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.125.190:33823,127.18.125.189:44967,127.18.125.188:38587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/krb5kdc/kudu_127.18.125.188.keytab
--principal=kudu/127.18.125.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.125.188:38587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.125.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:32.162254 10364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:32.163326 10364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:32.172325 10371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:32.172318 10370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:32.172313 10373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:32.172458 10364 server_base.cc:1034] running on GCE node
I20260806 18:39:32.173296 10364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:32.174652 10364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:32.175864 10364 hybrid_clock.cc:648] HybridClock initialized: now 1786041572175802 us; error 80 us; skew 500 ppm
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041572, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:32.179292 10364 init.cc:377] Logged in from keytab as kudu/127.18.125.188@KRBTEST.COM (short username kudu)
I20260806 18:39:32.180670 10364 webserver.cc:460] Webserver started at http://127.18.125.188:46721/ using document root <none> and password file <none>
I20260806 18:39:32.181370 10364 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:32.181447 10364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:32.181715 10364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:32.184310 10364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/data/instance:
uuid: "3b21386f968746ed8e776387143b65f9"
format_stamp: "Formatted at 2026-08-06 18:39:32 on dist-test-slave-8w9v"
server_key: "0f2fcc1d92f49544e0a289d4e99b79f3"
server_key_iv: "586aae8820dfcb791cfe3d15614e3e31"
server_key_version: "encryptionkey@0"
I20260806 18:39:32.185012 10364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/wal/instance:
uuid: "3b21386f968746ed8e776387143b65f9"
format_stamp: "Formatted at 2026-08-06 18:39:32 on dist-test-slave-8w9v"
server_key: "0f2fcc1d92f49544e0a289d4e99b79f3"
server_key_iv: "586aae8820dfcb791cfe3d15614e3e31"
server_key_version: "encryptionkey@0"
I20260806 18:39:32.189595 10364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 18:39:32.192644 10381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:32.194075 10364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:39:32.194255 10364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/data,/tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/wal
uuid: "3b21386f968746ed8e776387143b65f9"
format_stamp: "Formatted at 2026-08-06 18:39:32 on dist-test-slave-8w9v"
server_key: "0f2fcc1d92f49544e0a289d4e99b79f3"
server_key_iv: "586aae8820dfcb791cfe3d15614e3e31"
server_key_version: "encryptionkey@0"
I20260806 18:39:32.194397 10364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:32.201392 10278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } attempt: 1
I20260806 18:39:32.202257 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.201969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:44036)
0806 18:39:32.202118 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.202179 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 18:39:32.203110 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.202886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:44038)
0806 18:39:32.202992 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.203045 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 18:39:32.203356 10278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.125.188:38587: Network error: Client connection negotiation failed: client connection to 127.18.125.188:38587: connect: Connection refused (error 111)
I20260806 18:39:32.211169 10364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:32.215354 10364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:32.215600 10364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:32.225198 10364 rpc_server.cc:307] RPC server started. Bound to: 127.18.125.188:38587
I20260806 18:39:32.225540 10436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.125.188:38587 every 8 connection(s)
I20260806 18:39:32.226462 10364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/data/info.pb
I20260806 18:39:32.229883 10438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:32.233568 18934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskun4LBX/build/debug/bin/kudu as pid 10364
I20260806 18:39:32.233724 18934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskun4LBX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041269054381-18934-0/minicluster-data/master-2/wal/instance
I20260806 18:39:32.234063 18934 external_mini_cluster.cc:1428] Setting key 2505e637b8debf6eca88a3fec3b153d9
I20260806 18:39:32.234372 10438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041572, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
I20260806 18:39:32.247609 10357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } attempt: 3
I20260806 18:39:32.248267 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.236628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57452 (local address 127.18.125.190:33823)
0806 18:39:32.236764 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.236772 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.237023 (+ 251us) server_negotiation.cc:407] Connection header received
0806 18:39:32.237423 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.237426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.237489 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.237574 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.238419 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.238942 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.240014 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.240379 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.240560 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.243606 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.243609 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.243611 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.243641 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.245753 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.246348 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.246352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.246353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.246446 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.246787 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.246790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.246792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.247020 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.247136 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.247886 (+ 750us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.248063 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:39:32.250164 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.235609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44044 (local address 127.18.125.188:38587)
0806 18:39:32.236174 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.236192 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.236238 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:39:32.237188 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.237213 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.237436 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.237626 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.238478 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.239347 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.240265 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.240765 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.240879 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.243864 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.243870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.243883 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.243907 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.245750 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.246281 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.246287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.246288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.246402 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.246700 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.246704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.246706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.247650 (+ 944us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.248009 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.248829 (+ 820us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.249000 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":342,"thread_start_us":150,"threads_started":1}
I20260806 18:39:32.250164 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.236432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.190:33823 (local address 127.0.0.1:57452)
0806 18:39:32.236885 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.236926 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:32.237237 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:32.237640 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:32.237645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:32.238015 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:32.238272 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.238285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.239055 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.239060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.239821 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.239830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.240596 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.240600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.240777 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.241447 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:32.241474 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:32.243356 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:32.245926 (+ 2570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.245935 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.245948 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.246206 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.246546 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.246549 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.246551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.246683 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.247137 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:32.247147 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:32.247473 (+ 326us) client_negotiation.cc:769] Sending connection context
0806 18:39:32.247719 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 18:39:32.248943 (+ 1224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":4,"spinlock_wait_cycles":7040,"thread_start_us":145,"threads_started":1}
I20260806 18:39:32.251868 10438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.189@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041572, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.190@KRBTEST.COM
I20260806 18:39:32.263248 10360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.248186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:44052)
0806 18:39:32.248433 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.248453 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:32.248556 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:32.249600 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:32.249604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:32.249787 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:32.249889 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.249895 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.251438 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.251443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.252188 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.252211 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.252876 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.252880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.253023 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.253658 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:32.253680 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:32.255605 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:32.259624 (+ 4019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.259628 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.259631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.259923 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.260604 (+ 681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.260607 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.260609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.260720 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.261774 (+ 1054us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:32.261778 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:32.262164 (+ 386us) client_negotiation.cc:769] Sending connection context
0806 18:39:32.262701 (+ 537us) client_negotiation.cc:240] Negotiation successful
0806 18:39:32.262907 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260806 18:39:32.264170 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.248229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.18.125.188:38587)
0806 18:39:32.249340 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.249348 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.249359 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:39:32.249389 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.249391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.249446 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.249558 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.250018 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.251222 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.252340 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.252693 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.253316 (+ 623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.255772 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.255778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.255782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.255813 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.259524 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.260319 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.260328 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.260330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.260465 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.260921 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.260928 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.260932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.261549 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.261793 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.263722 (+ 1929us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.263911 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1048,"thread_start_us":151,"threads_started":1}
I20260806 18:39:32.265524 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.252520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.189:44967 (local address 127.0.0.1:46556)
0806 18:39:32.252827 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.252845 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:32.253023 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:32.253426 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:32.253428 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:32.253600 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:32.253698 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.253703 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.254880 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.254886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.255616 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.255628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.256424 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.256430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.256612 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.257243 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:32.257257 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:32.258985 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:32.262460 (+ 3475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.262463 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.262466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.262677 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.263255 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.263282 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.263285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.263535 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.264811 (+ 1276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:32.264813 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:32.264914 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:39:32.265180 (+ 266us) client_negotiation.cc:240] Negotiation successful
0806 18:39:32.265362 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260806 18:39:32.266518 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.253992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57466 (local address 127.18.125.190:33823)
0806 18:39:32.254129 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.254133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.254241 (+ 108us) server_negotiation.cc:407] Connection header received
0806 18:39:32.254392 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.254395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.254448 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.254554 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.255496 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.256388 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.257601 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.257956 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.258087 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.261493 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.261500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.261503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.261539 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.263615 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.264283 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.264287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.264291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.264395 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.264736 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.264739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.264741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.265073 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.265379 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.266270 (+ 891us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.266402 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:39:32.268548 10357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7: Bootstrap starting.
I20260806 18:39:32.270576 10278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } attempt: 2
I20260806 18:39:32.271629 10363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.252669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46556 (local address 127.18.125.189:44967)
0806 18:39:32.252849 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.252854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.252916 (+ 62us) server_negotiation.cc:407] Connection header received
0806 18:39:32.253148 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.253152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.253204 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.253263 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.253947 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.254756 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.255798 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.256273 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.256386 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.259186 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.259193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.259197 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.259232 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.262336 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.262794 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.262798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.262800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.262892 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.263958 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.263964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.263966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.264628 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.264748 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.271196 (+ 6448us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.271381 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:39:32.271445 10357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:32.272483 10438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:39:32.272917 10357 log.cc:824] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:32.276104 10357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7: No bootstrap required, opened a new log
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.190@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041572, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.125.188@KRBTEST.COM for kudu/127.18.125.188@KRBTEST.COM
Aug 06 18:39:32 dist-test-slave-8w9v krb5kdc[10191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.125.189@KRBTEST.COM
I20260806 18:39:32.281862 10357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } }
I20260806 18:39:32.282271 10357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43f5c40f21044cb0a8eb76b214eceac7, State: Initialized, Role: FOLLOWER
I20260806 18:39:32.282740 10357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } }
I20260806 18:39:32.282927 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.271154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:44056)
0806 18:39:32.271386 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.271403 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:32.271501 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:32.271842 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:32.271848 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:32.272085 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:32.272274 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.272287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.273996 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.274000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.274886 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.274899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.275614 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.275618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.275716 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.276407 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:32.276425 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:32.278220 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:32.280529 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.280538 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.280543 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.280882 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.281345 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.281349 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.281351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.281464 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.282259 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:32.282263 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:32.282400 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 18:39:32.282511 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 18:39:32.282688 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260806 18:39:32.283818 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.271301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44056 (local address 127.18.125.188:38587)
0806 18:39:32.271438 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.271443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.271460 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:39:32.271618 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.271621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.271676 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.272874 (+ 1198us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.272993 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.273819 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.275079 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.275457 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.276016 (+ 559us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.278378 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.278382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.278385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.278412 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.280289 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.281080 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.281086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.281089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.281202 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.281601 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.281608 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.281613 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.282066 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.283350 (+ 1284us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.283445 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.283593 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:39:32.284054 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.273098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.125.188:38587 (local address 127.0.0.1:44070)
0806 18:39:32.273601 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:32.273612 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:32.273727 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:32.274001 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:32.274003 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:32.274201 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:32.274324 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.274333 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.275420 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.275423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.276222 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:32.276235 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.276836 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.276842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:32.276924 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.277424 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:32.277438 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:32.280102 (+ 2664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:32.282531 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.282534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.282536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.282768 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.283062 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:32.283066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:32.283069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:32.283147 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:32.283582 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:32.283585 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:32.283664 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 18:39:32.283755 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 18:39:32.283879 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465}
I20260806 18:39:32.284607 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.273334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.18.125.188:38587)
0806 18:39:32.273551 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.273554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.273687 (+ 133us) server_negotiation.cc:407] Connection header received
0806 18:39:32.273830 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.273834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.273880 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.274461 (+ 581us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.274546 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.275281 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.276360 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.276708 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.277141 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.280270 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.280274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.280276 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.280300 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.282420 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.282879 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.282882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.282884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.282965 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.283232 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.283235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.283237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.283477 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.284285 (+ 808us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.284375 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.284486 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260806 18:39:32.285419 10357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:32.285818 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.268074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46562 (local address 127.18.125.189:44967)
0806 18:39:32.271689 (+ 3615us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.271699 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.271714 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:39:32.271755 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.271758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.271808 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.272841 (+ 1033us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.272957 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.273849 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.275069 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.275475 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.276279 (+ 804us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.279817 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.279821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.279825 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.279865 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.282922 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.283553 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.283558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.283559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.283659 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.284082 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.284085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.284087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.284375 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.285342 (+ 967us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.285435 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.285608 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3535,"thread_start_us":212,"threads_started":1}
I20260806 18:39:32.287447 10278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1: Bootstrap starting.
I20260806 18:39:32.289541 10446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } } }
I20260806 18:39:32.289671 10446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:32.290946 18934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 18:39:32.291409 10457 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:32.291502 10457 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:32.290313 10285 master_runner.cc:186] Error getting master registration for 127.18.125.190:33823: OK, instance_id { permanent_uuid: "afd4c5d989d844edbb306421897589b1" instance_seqno: 1786041571729529 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:39:32.293618 10285 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:39:32.292299 10252 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "afd4c5d989d844edbb306421897589b1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57466
I20260806 18:39:32.294713 10278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:32.295650 10278 log.cc:824] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:32.296519 10438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9: Bootstrap starting.
I20260806 18:39:32.298354 10278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1: No bootstrap required, opened a new log
I20260806 18:39:32.299363 10438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:32.300320 10438 log.cc:824] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:32.301687 10278 raft_consensus.cc:348] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } }
I20260806 18:39:32.301944 10278 raft_consensus.cc:729] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afd4c5d989d844edbb306421897589b1, State: Initialized, Role: FOLLOWER
I20260806 18:39:32.302301 10278 consensus_queue.cc:260] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } }
I20260806 18:39:32.302666 10438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9: No bootstrap required, opened a new log
I20260806 18:39:32.303059 10461 sys_catalog.cc:455] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } } }
I20260806 18:39:32.303966 10278 sys_catalog.cc:564] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:32.304327 10252 raft_consensus.cc:482] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 18:39:32.304467 10252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:32.305886 10252 raft_consensus.cc:504] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } }
I20260806 18:39:32.307873 10328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "afd4c5d989d844edbb306421897589b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "43f5c40f21044cb0a8eb76b214eceac7"
I20260806 18:39:32.308209 10328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:32.308298 10438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } }
I20260806 18:39:32.308581 10438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b21386f968746ed8e776387143b65f9, State: Initialized, Role: FOLLOWER
I20260806 18:39:32.308936 10438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } }
I20260806 18:39:32.309600 10328 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afd4c5d989d844edbb306421897589b1 in term 1.
I20260806 18:39:32.309944 10252 leader_election.cc:288] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [CANDIDATE]: Term 1 election: Requested vote from peers 43f5c40f21044cb0a8eb76b214eceac7 (127.18.125.189:44967), 3b21386f968746ed8e776387143b65f9 (127.18.125.188:38587)
I20260806 18:39:32.310535 10224 leader_election.cc:302] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43f5c40f21044cb0a8eb76b214eceac7, afd4c5d989d844edbb306421897589b1; no voters:
I20260806 18:39:32.311337 10410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "afd4c5d989d844edbb306421897589b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3b21386f968746ed8e776387143b65f9"
I20260806 18:39:32.311574 10410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:32.303894 10461 sys_catalog.cc:458] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:32.312383 10461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:32.312511 10461 raft_consensus.cc:686] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [term 1 LEADER]: Becoming Leader. State: Replica: afd4c5d989d844edbb306421897589b1, State: Running, Role: LEADER
I20260806 18:39:32.312896 10461 consensus_queue.cc:237] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } }
I20260806 18:39:32.313105 10410 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afd4c5d989d844edbb306421897589b1 in term 1.
I20260806 18:39:32.313851 10438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:32.315336 10461 sys_catalog.cc:455] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afd4c5d989d844edbb306421897589b1. Latest consensus state: current_term: 1 leader_uuid: "afd4c5d989d844edbb306421897589b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } } }
I20260806 18:39:32.315557 10461 sys_catalog.cc:458] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:32.316107 10488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:32.314009 10205 master_runner.cc:186] Error getting master registration for 127.18.125.188:38587: OK, instance_id { permanent_uuid: "3b21386f968746ed8e776387143b65f9" instance_seqno: 1786041572222641 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:39:32.318626 10205 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 18:39:32.322865 10494 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:32.323123 10494 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:32.323829 10475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } } }
I20260806 18:39:32.324174 10475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:32.326529 10488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:32.335217 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.317889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.18.125.188:38587)
0806 18:39:32.318035 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.318041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.324734 (+ 6693us) server_negotiation.cc:407] Connection header received
0806 18:39:32.324929 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.324932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.324999 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.325087 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.327440 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.328263 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.329366 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.329660 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.329738 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.331436 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.331440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.331443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.331470 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.333286 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.333786 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.333790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.333791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.333880 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.334194 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.334200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.334203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.334444 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.334556 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.334923 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.335052 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:39:32.335554 10410 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [term 1 FOLLOWER]: Refusing update from remote peer afd4c5d989d844edbb306421897589b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:32.336472 10328 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [term 1 FOLLOWER]: Refusing update from remote peer afd4c5d989d844edbb306421897589b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:32.336727 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.318279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57470 (local address 127.18.125.190:33823)
0806 18:39:32.318402 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.318407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.319016 (+ 609us) server_negotiation.cc:407] Connection header received
0806 18:39:32.319233 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.319236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.319290 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.319345 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.320866 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.321767 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.323735 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.324162 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.324275 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.326614 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.326618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.326621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.326652 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.328465 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.332256 (+ 3791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.332261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.332263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.332667 (+ 404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.333042 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.333048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.333050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.333301 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.333433 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.336449 (+ 3016us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.336577 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:39:32.338141 10461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:32.342964 10461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:32.344282 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:32.317441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.18.125.189:44967)
0806 18:39:32.317574 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.317579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.334243 (+ 16664us) server_negotiation.cc:407] Connection header received
0806 18:39:32.334296 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.334300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.334355 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.334433 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.335918 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.336749 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.337833 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:32.338157 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:32.338286 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:32.339888 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:32.339892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:32.339894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:32.339928 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:32.341832 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.342323 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.342326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.342328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.342432 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:32.342717 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:32.342721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:32.342722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:32.342982 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:32.343881 (+ 899us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:32.343961 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 18:39:32.344091 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:39:32.348105 10446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afd4c5d989d844edbb306421897589b1. Latest consensus state: current_term: 1 leader_uuid: "afd4c5d989d844edbb306421897589b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } } }
I20260806 18:39:32.348335 10446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:32.348490 10475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afd4c5d989d844edbb306421897589b1. Latest consensus state: current_term: 1 leader_uuid: "afd4c5d989d844edbb306421897589b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } } }
I20260806 18:39:32.348605 10475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:32.355206 10461 sys_catalog.cc:455] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afd4c5d989d844edbb306421897589b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } } }
I20260806 18:39:32.355372 10461 sys_catalog.cc:458] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:32.358994 10475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afd4c5d989d844edbb306421897589b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } } }
I20260806 18:39:32.359151 10475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:32.359272 10502 mvcc.cc:205] Tried to move back new op lower bound from 7315626280279367680 to 7315626280198586368. Current Snapshot: MvccSnapshot[applied={T|T < 7315626280279367680}]
I20260806 18:39:32.359478 10501 mvcc.cc:205] Tried to move back new op lower bound from 7315626280279367680 to 7315626280198586368. Current Snapshot: MvccSnapshot[applied={T|T < 7315626280279367680}]
I20260806 18:39:32.361341 10461 sys_catalog.cc:455] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "afd4c5d989d844edbb306421897589b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } } }
I20260806 18:39:32.361488 10461 sys_catalog.cc:458] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:32.361842 10488 catalog_manager.cc:1347] Generated new cluster ID: de1098d6268941bdb6ba9e0dae579f3e
I20260806 18:39:32.361908 10488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:32.362843 10446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "afd4c5d989d844edbb306421897589b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afd4c5d989d844edbb306421897589b1" member_type: VOTER last_known_addr { host: "127.18.125.190" port: 33823 } } peers { permanent_uuid: "43f5c40f21044cb0a8eb76b214eceac7" member_type: VOTER last_known_addr { host: "127.18.125.189" port: 44967 } } peers { permanent_uuid: "3b21386f968746ed8e776387143b65f9" member_type: VOTER last_known_addr { host: "127.18.125.188" port: 38587 } } }
I20260806 18:39:32.362962 10446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:39:32.389809 10488 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:32.390909 10488 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:32.418988 10488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afd4c5d989d844edbb306421897589b1: Generated new TSK 0
I20260806 18:39:32.421371 10488 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:39:32.459170 10281 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.456266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57484 (local address 127.18.125.190:33823)
0806 18:39:32.456415 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.456420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.457872 (+ 1452us) server_negotiation.cc:407] Connection header received
0806 18:39:32.458112 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.458116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.458175 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.458244 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.459011 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57484: BlockingRecv error: recv got EOF from 127.0.0.1:57484 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 18:39:32.461022 10443 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.457244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.18.125.188:38587)
0806 18:39:32.457418 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.457424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.459942 (+ 2518us) server_negotiation.cc:407] Connection header received
0806 18:39:32.460186 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.460190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.460246 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.460303 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.460860 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44080: BlockingRecv error: recv got EOF from 127.0.0.1:44080 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 18:39:32.463476 10444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.462288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.18.125.189:44967)
0806 18:39:32.462448 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.462453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.462472 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:39:32.462638 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.462642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.462695 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.462749 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.463340 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46580: BlockingRecv error: recv got EOF from 127.0.0.1:46580 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 18:39:32.468765 10444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.466864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46592 (local address 127.18.125.189:44967)
0806 18:39:32.467024 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.467029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.467617 (+ 588us) server_negotiation.cc:407] Connection header received
0806 18:39:32.467866 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.467870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.467972 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.468032 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.468640 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46592: BlockingRecv error: recv got EOF from 127.0.0.1:46592 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 18:39:32.474552 10444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.472807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46602 (local address 127.18.125.189:44967)
0806 18:39:32.472924 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.472941 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.473468 (+ 527us) server_negotiation.cc:407] Connection header received
0806 18:39:32.473675 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.473678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.473736 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.473790 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.474432 (+ 642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46602: BlockingRecv error: recv got EOF from 127.0.0.1:46602 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:39:32.481026 10444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.479205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46614 (local address 127.18.125.189:44967)
0806 18:39:32.479323 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.479327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.479890 (+ 563us) server_negotiation.cc:407] Connection header received
0806 18:39:32.480158 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.480161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.480221 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.480284 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.480900 (+ 616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46614: BlockingRecv error: recv got EOF from 127.0.0.1:46614 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 18:39:32.490515 10444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.488677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.18.125.189:44967)
0806 18:39:32.488839 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.488844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.489396 (+ 552us) server_negotiation.cc:407] Connection header received
0806 18:39:32.489647 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.489651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.489710 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.489770 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.490376 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46620: BlockingRecv error: recv got EOF from 127.0.0.1:46620 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260806 18:39:32.503047 10444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.501244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46630 (local address 127.18.125.189:44967)
0806 18:39:32.501374 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.501378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.502004 (+ 626us) server_negotiation.cc:407] Connection header received
0806 18:39:32.502210 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.502214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.502269 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.502321 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.502915 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46630: BlockingRecv error: recv got EOF from 127.0.0.1:46630 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 18:39:32.523484 10444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.521911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46646 (local address 127.18.125.189:44967)
0806 18:39:32.522047 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.522053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.522562 (+ 509us) server_negotiation.cc:407] Connection header received
0806 18:39:32.522737 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.522741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.522803 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.522898 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.523364 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46646: BlockingRecv error: recv got EOF from 127.0.0.1:46646 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 18:39:32.559831 10444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.558406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46648 (local address 127.18.125.189:44967)
0806 18:39:32.558543 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.558548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.558831 (+ 283us) server_negotiation.cc:407] Connection header received
0806 18:39:32.558976 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.558980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.559040 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.559151 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.559696 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46648: BlockingRecv error: recv got EOF from 127.0.0.1:46648 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 18:39:32.627416 10444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.626133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46664 (local address 127.18.125.189:44967)
0806 18:39:32.626309 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.626314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.626476 (+ 162us) server_negotiation.cc:407] Connection header received
0806 18:39:32.626657 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.626661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.626720 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.626813 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.627305 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46664: BlockingRecv error: recv got EOF from 127.0.0.1:46664 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 18:39:32.759222 10444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:32.758012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46678 (local address 127.18.125.189:44967)
0806 18:39:32.758218 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:32.758221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:32.758327 (+ 106us) server_negotiation.cc:407] Connection header received
0806 18:39:32.758448 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:32.758451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:32.758515 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:32.758603 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:32.759095 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46678: BlockingRecv error: recv got EOF from 127.0.0.1:46678 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
W20260806 18:39:33.019541 10444 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:33.018074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46680 (local address 127.18.125.189:44967)
0806 18:39:33.018232 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:33.018237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:33.018442 (+ 205us) server_negotiation.cc:407] Connection header received
0806 18:39:33.018645 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:33.018649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:33.018730 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:33.018829 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:33.019388 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46680: BlockingRecv error: recv got EOF from 127.0.0.1:46680 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:39:33.292989 10457 catalog_manager.cc:1259] Loaded cluster ID: de1098d6268941bdb6ba9e0dae579f3e
I20260806 18:39:33.293088 10457 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7: loading cluster ID for follower catalog manager: success
I20260806 18:39:33.294930 10457 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7: acquiring CA information for follower catalog manager: success
I20260806 18:39:33.296217 10457 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 43f5c40f21044cb0a8eb76b214eceac7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:39:33.324645 10494 catalog_manager.cc:1259] Loaded cluster ID: de1098d6268941bdb6ba9e0dae579f3e
I20260806 18:39:33.324753 10494 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9: loading cluster ID for follower catalog manager: success
I20260806 18:39:33.326543 10494 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9: acquiring CA information for follower catalog manager: success
I20260806 18:39:33.327754 10494 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b21386f968746ed8e776387143b65f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:39:33.539677 10580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:33.534563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49774 (local address 127.18.125.189:44967)
0806 18:39:33.534848 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:33.534852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:33.534868 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:39:33.535045 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:33.535048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:33.535263 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:33.535584 (+ 321us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:33.535893 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:33.537045 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:33.537979 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:33.538234 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:33.538353 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:33.538717 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:33.539211 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:33.539371 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:33.539434 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 18:39:33.539502 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":92,"threads_started":1}
I20260806 18:39:33.547260 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:33.543256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43806 (local address 127.18.125.188:38587)
0806 18:39:33.543492 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:33.543496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:33.543510 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:39:33.543600 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:33.543602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:33.543729 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:33.543840 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:33.544284 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:33.544841 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:33.545680 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:33.545897 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:33.545982 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:33.546331 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:33.546813 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:33.546910 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:33.547064 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 18:39:33.547123 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":90,"threads_started":1}
I20260806 18:39:33.555408 10580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:33.550783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.18.125.189:44967)
0806 18:39:33.550929 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:33.550934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:33.551193 (+ 259us) server_negotiation.cc:407] Connection header received
0806 18:39:33.551484 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:33.551488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:33.551546 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:33.551624 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:33.552185 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:33.552991 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:33.554028 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:33.554373 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:33.554492 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:33.554851 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:33.554952 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:33.555042 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:33.555175 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 18:39:33.555253 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:39:33.556154 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:33.551255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43822 (local address 127.18.125.188:38587)
0806 18:39:33.551370 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:33.551377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:33.551391 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:39:33.551473 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:33.551476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:33.551530 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:33.551622 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:33.553130 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:33.553799 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:33.554911 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:33.555167 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:33.555263 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:33.555637 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:33.555704 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:33.555742 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:33.555908 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 18:39:33.555992 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:39:33.556571 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:33.550844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42634 (local address 127.18.125.190:33823)
0806 18:39:33.551351 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:33.551355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:33.551960 (+ 605us) server_negotiation.cc:407] Connection header received
0806 18:39:33.552190 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:33.552193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:33.552372 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:33.552504 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:33.552707 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:33.553493 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:33.554459 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:33.554711 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:33.554832 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:33.555447 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:33.555863 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:33.555937 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:33.556367 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 18:39:33.556434 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":92,"threads_started":1}
I20260806 18:39:33.560271 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 10205
I20260806 18:39:33.569201 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 10285
I20260806 18:39:33.576681 18934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskun4LBX/build/debug/bin/kudu with pid 10364
2026-08-06T18:39:33Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3270 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6678 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (304514 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST